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16.  Abstract 

In  1993  the  Federal  Aviation  Administration  (FAA)  began  deploying  two  new  wind  shear  detection  systems;  the 
Terminal  Doppler  Weather  Radar  (TDWR)  and  the  third-generation  Low  Level  Windshear  Alert  System 
(LLWAS  3),  Currently,  9  airports  are  scheduled  to  receive  both  a  TDWR  and  an  LLWAS  3.  This  number  may 
eventually  increase  to  as  high  as  45.  When  co-located  the  systems  will  be  integrated  to  provide  a  single  set  of  wind 
shear  alerts  and  improve  system  performance. 

The  TDWR  production  schedule  recpiired  one  of  three  integration  algorithms  to  be  chosen  for  speciflcation  by 
fall  1991.  The  three  algorithms  are  the  prototype  integration  algorithm  developed  at  the  National  Center  for 
Atmospheric  Research  (NCAR)  and  the  two  algorithms  developed  at  MIT  Lincoln  Laboratory  (MIT  LL).  To  assess 
the  performance  of  the  three  algorithms,  MIT  LL  performed  a  study  of  integration,  TDWH,  and  LLWAS  3  algo¬ 
rithms  at  Orlando  International  Airport  (MCO)  in  the  summer  of  1991.  Based  on  the  results  of  this  study,  MIT  LL 
and  NCAR  issued  a  joint  recommendation  that  the  FAA  procure  one  of  the  integration  algorithms  developed  at  MIT 
LL.  This  algorithm  was  demonstrated  at  the  Orlando  International  Airport  in  the  summer  of  1992. 

Results  of  the  1991  comparative  study  and  a  followup  study  of  the  TDWR,  LLWAS  3,  and  Message  Level 
integration  algorithms  at  Orlando  in  1992  are  discussed.  All  the  algorithms  met  the  requirement  of  detecting 
90  percent  of  microburst  level  wind  shear  with  loss  events.  LLWAS  3,  Build  5  TDWR,  and  the  MIT  LL  integration 
algorithms  with  Build  5  TDWR  all  met  the  requirement  that  less  than  10  percent  of  wind  shear  alerts  be  false.  The 
NCAR  prototype  did  not  utilize  Build  3  TD\^.  Build  4  TDWR  and  all  integration  algorithms  run  with  Build  4 
TDWR  did  not  meet  the  false-alert  requirement.  Detailed  descriptions  of  the  algorithms  are  given.  The  methodology 
for  estimating  various  algorithm  performance  statistics  based  on  a  comparison  with  a  dual-Doppler  algorithm  is 
detailed. 
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In  1 993  the  Federal  Aviation  Administration  (FAA)  began  deploying  two  new  wind  shear  detec¬ 
tion  systems;  the  Terminal  Doppler  Weather  Radar  (TDWR)  and  the  third-generation  Low  Level 
Windshear  Alen  System  (LLWAS  3).  Currently,  nine  airports  are  scheduled  to  receive  both  a  TDWR 
and  an  LLWAS  3.  This  number  may  eventually  increase  to  as  high  as  45.  When  co-located,  the  sys¬ 
tems  will  be  integrated  to  provide  a  single  set  of  wind  shear  alerts  and  improve  system  performance. 


The  TDWR  production  schedule  required  one  of  three  integration  algorithms  to  be  chosen  for 
specification  by  fall  1991.  The  three  algorithms  are  the  prototype  integration  algorithm  developed 
at  the  National  Center  for  Atmospheric  Research  (NCAR)  and  two  algorithms  developed  at  MIT 
Lincoln  Laboratory  (MIT/LL).  To  assess  the  performance  of  the  three  algorithms,  MIT/LL  per¬ 
formed  a  study  of  the  integration,  TDWR,  and  LLWAS  3  algorithms  at  Orlando  International  Airport 
(MCO)  in  the  Summer  of  1991.  Based  on  the  results  of  this  study,  Lincoln  Laboratory  and  NCAR 
issued  a  joint  recommendation  that  the  FAA  procure  one  of  the  integration  algorithms  developed 
at  MIT/LL.  This  algorithm  was  demonstrated  at  the  Orlando  International  Aiiport  in  the  summer 
of  1992. 

We  discuss  results  of  the  1991  comparative  study  and  a  follow-up  study  of  the  TDWR, 
LLWAS  3,  and  Message  Level  integration  algorithms  at  Orlando  in  1992.  All  of  the  algorithms  met 
the  requirement  of  detecting  90  percent  of  microburst  level  wind  shear  with  loss  events.  LLWAS  3, 
Build  5  TDWR,  and  the  MIT/LL  integration  algorithms  run  with  Build  5  TDWR,  all  met  the  require¬ 
ment  that  less  than  10  percent  of  wind  shear  alerts  be  false.  The  NCAR  prototype  did  not  utilize  Build 
5  TDWR.  Build  4  TDWR  and  all  the  integration  algorithms  run  with  Build  4  TDWR  did  not  meet 
the  false-alert  requirement.  Detailed  descriptions  of  the  algorithms  are  given.  The  methodology  for 
estimating  various  algorithm  performance  statistics  based  on  a  comparison  with  a  dual-Doppler  al¬ 
gorithm  is  detailed. 
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1.  INTRODUCTION 


In  1993  the  Federal  Aviation  Administration  (FAA)  began  deploying  two  new  wind  shear  detec¬ 
tion  systems:  the  Terminal  Doppler  Weather  Radar  (TDWR)  [6]  and  the  third-generation.  Low  Lev¬ 
el  Windshear  Alert  System  (LLWAS  3)  [8].  Currently  nine  airports  are  scheduled  to  receive  both 
a  TDWR  and  an  LLWAS  3.  This  number  may  eventually  increase  to  as  high  as  45.  When  co-located, 
the  systems  will  be  integrated  to  provide  a  single  set  of  wind  shear  alerts  and  improve  system  perfor¬ 
mance. 

The  TDWR  production  schedule  required  one  of  three  integration  algorithms  to  be  chosen  for 
specification  by  fall  1991.  The  three  algorithms  are  the  prototype  integration  algorithm  developed 
at  the  National  Center  for  Atmospheric  Research  (NCAR)  [5]  and  two  algorithms  developed  at  MIT 
Lincoln  Laboratory  (MIT/LL).  To  assess  the  performance  of  the  three  algorithms,  MIT/LL  per¬ 
formed  a  study  of  the  integration,  TDWR,  and  LLWAS  3  algorithms  at  Orlando  International  Airport 
(MCO)  in  the  summer  of  1991.  Based  on  the  results  of  this  study,  Lincoln  Laboratory  and  NCAR 
issued  a  joint  recommendation  that  the  FAA  procure  one  of  the  integration  algorithms  developed 
at  MIT/LL.  This  algorithm  was  demonstrated  at  the  Orlando  International  Aiipon  in  the  summer 
of  1992. 

This  repon  presents  the  results  of  the  1991  comparative  study  and  a  follow-up  study  of  the 
TDWR,  LLWAS  3,  and  Message  Level  integration  algorithms  on  1992  data  collected  at  MCO.  De¬ 
tailed  descriptions  of  the  algorithms  are  given,  followed  by  a  section  on  data  collection  at  the  Orlan¬ 
do  testbed.  Next,  the  methodology  for  estimating  various  algorithm  performance  statistics  based  on 
a  comparison  with  a  dual-Doppler  algorithm  is  detailed.  Lastly,  the  results  of  applying  this  method¬ 
ology  to  the  various  algorithms  are  presented  and  discussed  for  data  sets  collected  during  1991  and 
1992.  Results  of  a  study  on  the  real-time  operational  impact  of  TDWR,  LLWAS  3,  and  Message 
Level  integration  in  1992  at  MCO  are  presented  in  Appendix  A. 

The  results  presented  concern  only  the  detection  of  wind  shear  with  a  loss  of  head  wind,  consid¬ 
ered  the  primary  aviation  wind  shear  hazard.  While  important,  the  ability  to  detect  wind  shear  with 
a  gain  of  head  wind  was  not  a  determining  factor  in  the  comparison  since  each  integration  algorithm 
uses  the  same  logic  to  issue  gain  alerts. 
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2.  ALGORITHM  DESCRIPTIONS 


This  study  analyzes  the  performance  of  six  algorithms;  the  three  candidate  integration  algo¬ 
rithms,  TDWR  and  LLWAS  3  as  stand-alone  systems,  and  a  fourth  integration  algorithm  included 
to  aid  the  comparison.  Each  algorithm  produces  a  set  of  alphanumeric  runway  alerts.  Each  concrete 
runway  is  associated  with  four  operational  runways,  two  for  arrivals  and  two  for  departures,  and 
each  is  issued  a  separate  aleit.  Each  alert  contains  an  alert  type,  an  intensity  estimate,  and  a  location 
of  first  encounter. 

The  alert  types  are: 

•  Microburst  .Alert  (MBA),  a  wind  shear  with  a  loss  of  head  wind  of  30  knots 
or  greater  and 

•  Wind  Shear  Alert  (WSA),  a  wind  shear  with  a  loss  of  head  wind  of  at  least 
1 5  knots  and  less  than  30  knots,  or  a  gain  of  head  wind  of  1 5  knots  or  greater. 

The  intensity  is  the  loss  or  gain  in  head  wind  that  an  aircraft  flying  along  the  flight  path  is  ex¬ 
pected  to  experience,  rounded  to  the  nearest  5  knots. 

The  location  of  first  encounter  is  the  location  on  the  flight  path  where  an  aircraft  is  expected 
to  first  encounter  the  wind  shear  event.  The  location  of  first  encounter  is  given  as  on-the-runway, 
1  mile,  2  miles,  or  3  miles  from  runway  threshold.  The  maximum  alert  location  is  2  miles  for  depar¬ 
ture  runways  and  3  miles  for  arrival  runways. 

The  integration  algorithms  are  of  two  types:  message  level  and  product  level.  Message  level 
algorithms  integrate  the  alpha-numeric  runway  alert  messages.  Product  level  algorithms  integrate 
intermediate  algorithm  products  such  as  TDWR  microburst  shapes,  TDWR  features  aloft  [2],  and 
LLWAS  divergence  values.  The  integrated  products  are  used  to  generate  the  alphanumeric  alerts. 

The  four  integration  algorithms  are: 

•  Prototype  Product  Level  (PL-A), 

•  Product  Level  (PL-B), 

•  Message  Level  (ML),  and 

•  Union  (UN). 

The  integration  algorithms  vary  significantly  in  the  logic  used  to  generate  loss  alerts,  and  the 
logic  employed  by  each  is  discussed  in  detail  below.  All  of  the  integration  algorithms  issue  wind 
shear  alerts  with  a  gain  of  head  wind  using  the  same  logic.  LLWAS  provides  the  wind  shear  with 
gain  alerts  inside  its  coverage  region  and  TDWR  provides  them  outside  of  this  region. 

2.1.  TDWR 

TDWR  detects  wind  shear  by  analyzing  Doppler  radar  returns  from  an  area  covering  the  airport. 
Two  versions  of  the  TDWR  microburst  algorithm  were  used  to  generate  alerts  for  this  study.  The 
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first  is  the  algorithm  used  in  the  initial  TDWR  deployment  (Build  4),  and  the  second  is  an  upgrade 
to  the  first  deployment  (Build  5b).  The  Build  5  algorithm  includes  flight  path  shear  integration  to 
sharpen  the  accuracy  of  the  intensity  estimates.  The  TDWR  deployed  with  LLWAS  3  systems  uses 
flight  path  shear  integration.  The  non-shear-integration  method  is  included  in  this  study  because 
the  PL-A  algorithm  software  did  not  use  flight  path  shear  integration. 

The  TDWR  microburst  detection  algorithm  uses  four  processing  steps: 

1.  Build  loss  segments  by  examining  each  radial  of  Doppler  data  for  segments 
of  diverging  radial  velocities. 

2.  Group  loss  segments. 

3.  Fit  microburst  shapes  to  groups  of  loss  segments. 

4.  Intersect  microburst  shapes  with  runway  corridors  to  determine  runway 
location  and  loss  estimate. 

Without  flight  path  shear  integration  the  loss  estimate  for  a  given  runway  is  the  maximum  loss 
estimate  of  any  microburst  shape  that  intersects  its  runway  corridor.  When  flight  path  shear  integra¬ 
tion  is  used,  the  loss  estimate  is  computed  by  integrating  a  fractional  loss  from  each  microburst  shape 
along  the  runway  corridor.  This  fractional  loss  is  an  estimate  of  the  fraction  of  the  maximum  loss 
from  each  microburst  shape  that  an  aircraft  may  experience. 

In  addition  to  detecting  diverging  radial  velocities,  TDWR  also  detects  weather  features  aloft 
that  are  associated  with  microburst  activity.  These  features  include  both  Doppler  signamres,  such 
as  converging  winds  aloft,  and  reflectivity  signatures,  such  as  descending  storm  cores.  These  fea¬ 
tures  are  u.sed  to  reduce  the  time  that  it  takes  to  issue  an  alert.  If  these  features  are  not  present,  the 
microburst  algorithm  requires  a  detection  on  two  successive  radar  scans  before  issuing  an  alert.  If 
sufficient  features  aloft  are  present,  the  alert  is  issued  with  the  first  detection  of  wind  shear.  Featm  es 
aloft  play  an  important  role  in  the  product  level  integration  algorithms. 

2.2.  LLWAS  3 

LLWAS  3  detects  wind  shear  by  analyzing  wind  data  gathered  from  a  network  of  anemometers 
surrounding  the  airport  runways.  Divergences  and  convergences  in  the  surface  wind  field  are  esti¬ 
mated  from  the  wind  data  from  triples  and  pairs  of  anemometers.  Divergence  values  are  used  to  gen¬ 
erate  loss  alerts  and  convergence  values  are  used  to  generate  gain  alerts.  Detections  of  wind  field 
divergence  and  convergence  are  tested  for  statistical  significance  before  they  are  used  to  issue  an 
alert.  If  a  statistically  significant  divergence  is  sufficiently  large,  the  determination  is  made  that  a 
microburst  has  been  detected.  Due  to  the  anemometer  network  spacing,  it  is  likely  that  the  measured 
winds  do  not  reflect  the  maximum  outflow  strength.  Using  a  symmetry  hypothesis  and  knowledge 
of  the  network  geometry,  a  statistically  based  correction  factor  is  applied  to  reduce  the  effect  of  under 
measurement.  If  the  divergence  is  not  sufficiently  large  or  if  the  detection  is  a  convergence,  no 
correction  factor  is  applied.  In  either  case,  the  issued  alert  is  a  directly-measured  wind  shear  along 
the  direction  of  each  runway. 
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2.3.  Product  Level-A  (PL-A) 


The  PL-A  algorithm  is  the  prototype  product  level  integration  algorithm  developed  at  NCAR. 
This  algorithm  attempts  to  reduce  the  number  of  false  wind  shear  level  loss  alerts  by  dropping  weak 
wind  shear  level  LLWAS  loss  detections  that  are  not  near  additional  indications  of  hazardous  weath¬ 
er  (strong  TDWR  or  LLWAS  loss  detections  or  TDWR  features  aloft).  After  possibly  dropping  weak 
wind  shear  level  LLWAS  loss  detections,  the  algorithm  issues  the  strongest  alert  generated  from  ei¬ 
ther  LLWAS  or  TDWR  for  each  operational  runway.  If  one  system  is  generating  a  loss  alert  and  the 
other  is  generating  a  gain  alert,  the  loss  alert  is  issued.  A  prototype  of  this  algorithm  was  installed 
and  operated  at  Stapleton  International  Aiiport  in  Denver  from  1988  to  1991. 

The  logic  used  by  PL-A  to  convert  LLWAS  wind  field  divergence  values  into  runway  alerts 
-  differs  from  the  logic  used  in  LLWAS  3.  PL-A  scales  the  divergence  values  into  loss  estimates  and 
maps  them  to  a  regular  grid.  Next,  it  fits  TDWR  style  microburst  shapes  to  the  gridded  loss  estimates. 
Alens  are  then  generated  by  intersecting  the  shapes  with  the  runway  corridors  using  Build  4  TDWR 
logic. 

2.4.  Product  Level-B  (PL-B) 

The  PL-B  algorithm  is  a  product  level  algorithm  developed  at  MIT  Lincoln  Laboratory.  This 
algorithm  is  similar  to  PL-A,  but  uses  streamlined  processing,  and  attempts  to  reduce  a  greater  num¬ 
ber  of  false  wind  shear  alerts  as  well  as  false  microburst  alerts  from  both  LLWAS  and  TDWR.  Unlike 
PL-A,  this  algorithm  very  closely  follows  the  LLWAS  3  logic  for  converting  wind  field  divergence 
values  into  runway  alerts.  Like  the  PL-A  algorithm,  PL-B  drops  weak  wind  shear  alerts  from 
LLWAS  that  are  not  near  additional  indications  of  hazardous  weather.  Weak  wind  shear  alerts  from 
TDWR  are  treated  in  the  same  manner.  AdditicMially,  weak  microburst  alerts,  typically  alerts  with 
a  30  knot  loss  estimate,  that  are  not  near  indications  of  hazardous  weather  are  reduced  to  25  knot 
wind  shear  alens.  iltis  removes  a  microburst  alert  that  is  likely  to  be  false,  but  still  gives  an  indication 
to  the  pilot  that  some  level  of  wind  shear  may  be  present  in  case  the  alert  was  not  false.  TDWR  alerts 
for  events  lying  outside  the  region  of  solid  LLWAS  detection  coverage  are  not  subject  to  the  verifica¬ 
tion  and  modification  process. 

Algorithm  thresholds  are  set  for  individual  runways.  This  is  important  since  some  runways  have 
better  LLWAS  or  TDWR  coverage  than  others.  When  the  two  systems  issue  conflicting  alert  types, 
gain  vs.  loss  arbitration  logic  is  applied  to  determine  which  alert  to  issue.  When  the  two  systems  are 
issuing  loss  alerts,  the  strongest  alert  is  issued,  ^d  when  both  are  issuing  a  gain  alert,  the  LLWAS 
alert  is  issued  since  LLWAS  has  been  demonstrated  to  issue  more  credible  gain  alerts  than  TDWR. 
The  details  of  the  alert  arbitration  are  given  below  in  the  section  on  the  Message  Level  algorithm. 

2.5.  Message  Level  (ML) 

The  ML  algorithm  is  a  message  level  algorithm  developed  at  MIT  Lincoln  Laboratory  [3].  This 
algorithm  attempts  to  reduce  false  wind  shear  and  microburst  level  alerts  from  both  TDWR  and 
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LLWAS  3  in  much  the  same  way  that  PL-B  does.  Since  this  is  a  message  level  algorithm  the  only 
indications  of  hazardous  weather  are  the  TDWR  and  LLWAS  runway  alens.  Alens  from  the  two 
systems  are  compared  on  a  runway-by-runway  basis.  Weak  wind  shear  level  alerts  given  by  only 
one  system  are  dropped,  and  weak  microburst  alerts  given  by  only  one  system  are  reduced  to  wind 
shear  level.  Unlike  the  product  level  algorithms,  when  both  systems  are  issuing  a  loss  alert  the  inte¬ 
grated  loss  estimate  is  based  on  an  averaging  technique  to  sharpen  the  estimated  loss.  When  the  two 
systems  issue  conflicting  alert  types,  gain  vs.  loss  arbitration  logic  is  applied  to  determine  which  alert 
to  issue.  When  both  are  issuing  a  gain  alert,  the  LLWAS  alert  is  issued,  since  LLWAS  has  been  dem¬ 
onstrated  to  issue  more  credible  gain  alens  than  TDWR.  TDWR  alerts  for  events  lying  outside  the 
region  of  solid  LLWAS  detection  coverage  are  not  subject  to  the  verification  and  modification  pro¬ 
cess.  The  parameters  that  control  the  logic  to  reduce  false  alerts,  sharpen  loss  estimates,  and  arbitrate 
between  loss  and  gain  alerts  are  discussed  in  detail  in  [4]. 

2.5.1.  Reduction  of  False  Loss  Alerts 

There  are  three  types  of  “false”  loss  alerts: 

1 .  Wind  shear  with  loss  alerts  (WSA)  when  there  is  no  wind  shear  with  loss. 

2.  Microburst  alerts  (MBA)  when  there  is  no  wind  shear  with  loss. 

3.  Microburst  alerts  when  there  is  a  wind  shear  with  loss,  but  the  true  loss  value  is  below 
30  knots,  so  the  alert  should  have  been  issued  as  a  WSA. 

The  first  two  types  are  clearly  false  alerts.  The  third  type  of  false  alert  is  not,  strictly  speaking, 
a  false  alert;  it  is  an  over  warning.  The  reduction  of  strictly  false  loss  alerts  is  discussed  in  this  section. 
The  reduction  of  over  warning  is  accomplished  when  each  system  is  issuing  a  loss  alert  for  an  opera¬ 
tional  runway  and  the  integration  algorithm  joins  the  two  loss  estimates.  This  is  discussed  in  the  sec¬ 
tion  entitled  Joining  Loss  Alerts. 

Less  than  10  percent  of  the  WSA  with  loss  alerts  from  either  TDWR  or  LLWAS  are  false,  and 
both  systems  have  fairly  high  detection  rates  for  these  weak  loss  events.  This  allows  the  ML  algo¬ 
rithm  to  require  confirmation  of  weak  alerts.  For  a  given  operational  runway,  if  one  system  is  issuing 
a  WSA  with  loss  below  a  specified  threshold  that  is  within  coverage  of  the  other  system  and  the  other 
system  is  not  issuing  a  loss  alert,  the  alert  is  canceled.  The  thresholds  are  set  for  each  runway.  For 
example,  when  a  runway  has  poor  coverage  by  TDWR  due  to  a  poor  viewing  angle,  the  threshold 
for  requiring  weak  alerts  from  LLWAS  to  be  confirmed  by  TDWR  is  set  low.  This  is  done  because 
TDWR  will  miss  some  real  events  on  that  runway,  and  we  do  not  want  correct  LLWAS  alerts  can¬ 
celled  based  on  a  TDWR  missed  detection.  Since  there  is  always  a  chance  that  a  correct  alert  will 
be  canceled  due  to  a  missed  detection  by  the  other  system,  only  very  weak  alerts  are  subject  to  this 
confirmation  process. 

Weak  microburst  alerts  are  subject  to  a  similar  confirmation  process.  However,  the  cemse- 
quences  of  canceling  a  correct  alert  at  this  level  are  more  severe.  For  this  reason,  a  weak  microburst 
alert  that  is  not  confirmed  by  a  loss  alert  from  the  other  system  is  downgraded  to  a  25-knot  wind 
shear  with  loss  alert  instead  of  being  cancelled.  Again,  the  thresholds  are  set  for  each  runway. 
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2.5.2.  Alert  Arbitration  Between  Lx)ss  and  Gain  Alerts 


Alert  arbitration  is  used  to  decide  which  alert  to  issue  in  the  case  that  one  system  is  issuing  a 
loss  alert  and  the  other  is  issuing  a  gain  alert  for  the  same  operational  runway.  Two  guiding  principles 
are  used.  First,  loss  events  are  the  more  serious  safety  hazard  unless  the  gain  event  is  sufficiently 
stronger  than  the  loss  event.  If  the  loss  event  is  at  the  microburst  level,  the  loss  event  is  always  judged 
to  be  the  more  serious  safety  hazard.  Second,  changes  between  loss  alert  and  gain  alert  should  be 
minimized.  This  is  done  by  modifying  the  arbitration  logic  so  that  at  each  alert  time  it  is  easier  to 
issue  an  alert  of  the  same  type  as  the  last  alert  than  it  is  to  issue  an  alert  of  the  opposing  type. 

More  precisely,  suppose  we  have  both  a  loss  alert  and  a  gain  alert  for  an  op)erational  runway. 
If  the  loss  alert  is  a  microburst  alert  the  arbitration  issues  the  loss  alert.  If  the  loss  alert  is  at  the  wind 
shear  level,  there  are  three  cases. 

Case  1 :  Suppose  there  were  no  alert  for  the  last  alert  time.  Then  the  loss  alert  is  issued  unless 
the  gain  alert  is  stronger  than  the  loss  alert  by  15  knots  or  greater. 

Case  2:  Suppose  there  were  a  loss  alert  for  the  last  alert  time.  Then  the  loss  alert  is  issued 
unless  the  gain  alert  is  stronger  than  the  loss  alert  by  20  knots  or  greater.  That  is,  if  the  last 
alert  were  a  loss  alert  the  arbitration  makes  it  harder  for  a  gain  to  override  the  loss  than  in 
case  1.  This  reduces  the  likelihood  of  switching  alert  types  from  loss  to  gain. 

Case  3:  Suppose  there  were  a  gain  alert  for  the  last  polling  cycle.  Then  the  loss  alert  is  is¬ 
sued  unless  the  gain  alert  is  stronger  than  the  loss  alert  by  10  knots  or  greater.  In  this  case 
the  arbitration  makes  it  easier  to  continue  issuing  a  gain  alert,  or  rather,  it  makes  it  harder 
for  the  system  to  switch  alert  types  from  gain  to  loss. 

The  thresholds  in  the  three  cases  above  are  determined  by  a  basic  threshold;  in  this  case,  15 
knots.  This  threshold  is  adjusted  up  or  down  by  a  set  amount;  in  this  case  5  knots,  depending  on  the 
alert  that  was  issued  for  the  last  polling  cycle.  These  values  are  set  during  system  initialization.  They 
are  user  selectable,  and  these  are  the  current  recommended  values. 

2.53.  Joining  Loss  Alerts 

When  both  systems  are  issuing  loss  alerts  for  the  same  operational  runway,  the  integrated  loss 
estimate  is  an  average  of  the  two  values.  However,  care  must  be  taken  to  avoid  issuing  a  serious  un¬ 
derestimate  of  the  strength  of  the  event  should  one  system  issue  a  seriously  low  estimate.  While  it 
is  important  that  the  issued  loss  value  accurately  reflect  the  true  strength  of  an  event,  it  is  coisidered 
a  greater  safety  hazard  to  underestimate  an  event  than  it  is  to  overestimate  an  event.  Provided  the 
alert  locations  are  within  LLWAS  coverage,  the  integrated  loss  value  is  set  to; 

minimum <  average  loss  value,  Ki  x  LLWAS  loss  value,  K2  x  TDWR  loss  value}. 
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For  this  study,  the  values  of  Kj,  and  K2  are  set  to  0.8.  The  minimum  is  used  since  these  are  negative 
values.  This  means  that  the  average  value  is  issued  unless  the  average  is  less  than  80  percent  of  the 
largest  value  from  either  system.  The  integrated  loss  estimate  cannot  be  adjusted  downward  more 
than  20  percent  from  the  strongest  of  the  two  alerts.  If  the  TDWR  alen  location  is  outside  LLWAS 
coverage,  the  integrated  loss  estimate  is  the  maximum  of  the  two  alerts.  When  a  substantial  ponion 
of  the  outflow  is  outside  the  LLWAS  network,  it  is  not  deemed  safe  to  average  the  two  loss  estimates. 

2.5.4.  Joining  Gain  .Alerts 

The  mathematics  used  to  compute  the  integrated  gain  estimate  when  both  systems  are  issuing 
a  gain  alert  for  the  same  operational  runway  is  similar  to  the  mathematics  used  to  integrate  loss  esti¬ 
mates.  However,  the  the  parameters  are  set  to  achieve  an  estimate  using  a  very  different  logic.  Pro¬ 
vided  both  alerts  are  within  LLWAS  coverage,  the  integrated  gain  estimate  is  set  to; 

maximum  {average  gain  value,  Ki  x  LLWAS  gain  value,  K2  x  TDWR  gain  value}. 

Here,  Ki  is  set  to  one  and  K2  is  set  to  zero.  This  causes  the  LLWAS  gain  alert  to  always  be  chosen 
if  the  event  is  inside  LLWAS  coverage.  If  the  TDWR  gain  location  is  outside  LLWAS  coverage,  the 
integrated  gain  estimate  is  the  maximum  of  the  two  estimates.  The  mathematics  allows  the  algorithm 
to  incorporate  the  TDWR  gain  estimate  in  the  future  if  TDWR  is  upgraded  to  provide  more  reliable 
gain  estimates. 

2.6  Union 

The  Union  algorithm  is  a  message  level  algorithm  that  issues  the  strongest  alert  from  either  sys¬ 
tem  for  each  operational  runway.  Any  loss  alert  overrides  any  gain  alert,  and  no  attempt  is  made  to 
reduce  false  alerts  or  to  adjust  the  loss  or  gain  estimates.  This  simple  algorithm  is  used  as  a  baseline 
for  measuring  the  benefits  oi  the  other  integration  algorithms  as  they  increase  in  complexity  and  cost 
from  ML  to  PL-B  to  PL- A. 
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3.  TESTBED 


The  data  for  this  study  were  collected  at  the  Lincoln  Laboratory  testbed  at  the  Orlando  Intema- 
tioial  Airport  (MCO).  The  testbed  layout  is  shown  in  Figure  1. 


Figure  1.  1991  Orlando  testbed. 


3.1.  LLWASData 

The  LLWAS  data  were  collected  from  three  anemometer  networks:  six-sensor  LLWAS,  nine¬ 
sensor  LLWAS,  and  1 5-sensor  LL  mesonet.  The  six-sensor  LLWAS  network  is  the  Phase  II  LLWAS 
used  by  the  FAA  to  provide  wind  shear  detection  coverage  for  MCO.  The  six  commissioned  sensors 
were  moved  to  sites  chosen  for  the  LLWAS  3  and  placed  on  LLWAS  3  poles.  The  nine-sensor 
anemometer  network  is  a  non-commissioned  Phase  II  LLWAS  that  has  been  modified  to  poll  nine 
sensors.  It  consists  of  nine  sensors  that  are  to  be  added  to  the  original  six  sensors  to  complete  the 
LLWAS  3  for  MCO.  A  15-sensor  anemometer  network  oi  100-foot  poles  was  installed  by  MIT  Lin¬ 
coln  Laboratory  to  enlarge  the  coverage  region. 

The  asynchronous  data  from  the  three  networks  were  merged  into  ten-second  synchronous  ar¬ 
chive  records.  Each  record  contains  the  sensor  winds  at  all  30  sensors  for  a  lO-seccMid  time  period. 
The  resulting  data  recOTds  are  similar  to  the  data  records  in  the  LORAL  Data  Systems  LLWAS  IE. 
Each  record  contains  the  most  recent  data  from  each  sensor  during  the  previous  ten  seconds.  Missing 
and/or  corrupted  data  were  flagged  in  the  archive. 
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3.2.  TDWR  Data 

The  Lincoln  Laboratory  TDWR  testbed  radar  (FL-2C)  provided  the  TDWR  base  data.  The 
TDWR  products  needed  for  the  product  level  integration  algorithms  were  collected  during  normal 
FL-2C  operations.  TDWR  alerts  were  generated  using  the  TDWR  runway  alert  algorithm  both  with 
and  without  flight  path  shear  integration. 

The  TDWR  microburst  shapes  and  alert  values  needed  by  PL-A  were  generated  by  software 
provided  by  NCAR. 

3.3.  Dual-Doppler  Data 

The  radai  data  used  to  generate  the  dual-Doppler  wind  fields  were  collected  from  FL-2C  and 
the  University  of  North  Dakota  Doppler  radar  (UND).  The  FL-2C  radar  scanned  the  standard 
TDWR  coverage  region  mandated  for  MCO.  The  UND  scan  sector  was  chosen  to  completely  cover 
the  anemometer  network.  Dual-Doppler  data  were  available  in  1991  only.  In  1992,  the  UND 
Doppler  data  were  not  of  sufficient  quality  to  produce  dual-Doppler  wind  fields. 

3.4.  Weather  Summary 

It  is  important  to  have  enough  cases  so  that  the  evaluation  is  statistically  significant.  The  20  days 
used  in  this  study  were  chosen  because  they  contained  an  assortment  of  wind  shear  events,  from 
strong  microbursts  to  marginal  wind  shears.  They  also  had  complete  data,  allowing  a  good  set  of 
comparison  alerts  to  be  generated.  Tables  1  and  2  give  summary  accounts  of  the  wind  shear  events 
for  the  twenty  days.  The  approximate  numbers  of  runway  minutes  of  alerts  is  the  number  of  polls 
of  alerts  divided  by  six,  the  number  of  polls  per  minute.  If  a  10-minute  event  impacts  three  runways, 
there  are  30  runway  minutes  of  impact.  The  maximum  loss  is  the  maximum  loss  experienced  on  any 
runways  during  the  day  as  determined  from  the  comparison  alerts  (See  Section  4.). 

Table  1. 

1991  Weather  Summary 

(Date,  Approximate  Runway  Minutes  of  Alerts,  Maximum  Loss) 


Date 

MBA 

ws/ 

LOSS 

WS/ 

GAIN 

Max  Loss 

(knots) 

7/10/91 

27 

94 

18 

-66 

7/17/91 

70 

58 

42 

-44 

7/25/91 

0 

45 

213 

-29 

7/30/91 

0 

74 

0 

-36 

8/2/91 

35 

357 

57 

-40 

8/3/91 

3 

219 

5 

-45 

8/9/91 

245 

184 

58 

-89 

8/10/91 

64 

179 

29 

-62 

8/31/91 

258 

161 

83 

-52 

4 

349 

126 

-32 

Total 

706 

1720 

631 
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Table  2. 

1992  Weather  Summary 

(Date,  Approximate  Runway  Minutes  of  Alerts,  Maximum  Loss) 


Date 

MBA 

WS/ 

LOSS 

WS/ 

GAIN 

Max  Loss 

(knotsl 

6/2/92 

0 

244 

0 

-38 

6/3/92 

97 

539 

99 

-40 

6/12/92 

0 

39 

3 

-29 

6/21/92 

81 

369 

313 

-51 

6/30/92 

108 

496 

264 

-47 

7/7/92 

249 

484 

178 

-63 

7/19/92 

36 

651 

132 

-56 

8/1/92 

281 

803 

585 

-65 

8/3/92 

129 

562 

317 

-49 

8/14/92 

179 

359 

89 

-79 

Total 

1160 

4536 

1980 
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4.  EVALUATION  METHODOLOGY 


The  alert  type  and  loss/gain  estimate  from  each  algorithm  alen  was  compared  against  a  compar¬ 
ison  alert  generated  from  Doppler  measurements  along  the  flight  path  of  the  given  operational  run¬ 
way.  The  results  of  this  comparison  were  used  to  generate  performance  measures  such  as  probability 
of  detection,  probability  of  false  alert,  and  overall  system  accuracy.  In  1991 ,  the  comparison  alerts 
were  generated  from  a  dual-Doppler-based  wind  shear  detection  algorithm.  This  allowed  an  evalua¬ 
tion  for  runways  that  are  not  aligned  with  the  TDWR  line  of  sight.  In  1992,  comparison  aleits  were 
generated  in  a  manner  similar  to  1991,  but  due  to  UND  data  quality  problems,  only  TDWR  data 
were  used.  This  limited  the  1992  evaluation  to  runways  aligned  with  the  TDWR  line  of  site.  Compar¬ 
ison  alerts  from  either  the  dual-Doppler  algorithm  or  direct  Doppler  measurements  along  the  flight 
path,  while  not  perfect,  provide  a  good  estimate  of  the  actual  wind  shear  conditions. 

The  three  real  runways  at  MCO  cover  only  a  small  region  limiting  the  number  of  microburst 
impacts.  Furthermore,  TDWR  is  is  sited  to  look  directly  down  the  real  runways,  which  is  an  espe¬ 
cially  advantageous  situation  for  the  TDWR  algorithm.  In  1991, 14  imaginary  runways  were  laid 
out  in  the  region  covered  by  the  anemometer  network  to  capture  additional  microbui-st  impacts  and 
to  give  an  assortment  of  runways  at  different  angles  to  the  TDWR  line  of  sight.  In  1992,  two  imagi¬ 
nary  runways  oriented  north/south  were  used  in  addition  to  the  real  runways. 

4.1.  Dual  Doppler 

Dual-Doppler  alerts  are  constructed  in  three  steps: 

1 .  Compute  a  two-dimensional  wind  field  by  combining  observations  from  two 
radars. 

2.  Compute  runway  alerts  from  each  dual-Doppler  wind  field. 

3.  Interpolate  the  runway  alerts  in  time  to  produce  dual-Doppler  alerts  at  the 
time  of  the  algorithm  alerts. 

The  two-dimensional  wind  field  is  computed  using  standard  dual-Doppler  analysis  [1].  To  do 
this  processing,  the  observations  from  the  radars  must  be  nearly  simultaneous,  and  the  directions 
from  the  point  of  interest  to  the  two  radars  must  be  substantially  different.  Dual-Doppler  computa- 
tiois  are  done  for  each  point  in  a  grid  covering  the  area  of  interest.  The  grid  spacing  used  was  150 
meters. 

Once  the  two-dimensional  wind  field  has  been  computed,  loss  alerts  and  gain  alerts  are  com¬ 
puted  for  each  operational  runway  flight  path.  This  is  done  by  computing  the  runway  oriented  com¬ 
ponents  of  each  wind  vector  near  a  flight  path  and  using  these  components  to  find  the  maximum 
sustained  loss  and  the  maximum  sustained  gain  with  shears  above  specified  thresholds. 

Two  sets  of  dual-Doppler  alerts  are  computed.  One  set  is  computed  using  dual-Doppler  data 
points  within  a  narrow  (300  meter  wide)  corridor.  The  other  set  is  computed  using  dual-Doppler  data 
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points  within  a  wide  (1800  meter  wide)  coiridor.  Each  coiridor  covers  a  runway  and  extends  out 
from  the  runway  3  nm  for  arrival  runways  and  2  nm  for  departure  runways.  The  loss  alens  generated 
for  the  naiTow  corridor  require  a  shear  above  2.5  m/s/km  and  the  gain  alerts  require  a  shear  above 
1.9  m/s/km.  These  shear  thresholds  correspond  to  a  loss  of  20  knots  over  a  distance  of  4  km  and  a 
gain  of  15  knots  over  4  km,  which  are  the  standard  minimum  hazardous  shear  thresholds.  The  loss 
alerts  generated  for  the  wide  corridor  require  a  shear  above  2  m/s/km  and  the  gain  alens  require  a 
shear  above  1  m/s/km. 

The  viewpoint  of  the  study  is  that  a  dual-Doppler  alert  in  the  narrow  con  idor  must  be  matched 
by  an  algorithm  alert  and  that  an  algorithm  alert  is  not  considered  false  if  it  is  matched  by  a  dual- 
Doppler  alert  in  the  wide  corridor.  That  is,  if  the  dual-Doppler  algorithm  detects  a  loss  with  a  strong 
shear  on  the  flight  path,  an  algorithm  must  issue  an  alert  or  it  will  be  counted  as  missing  a  detection. 
Such  an  event  is  clearly  a  hazard  and  must  be  detected.  Conversely,  if  the  dual-Doppler  algorithm 
detects  a  loss  with  a  low  shear  or  one  that  is  well  to  the  side  of  the  center  line,  the  event  is  marginal 
and  an  algorithm  need  not  issue  an  alert.  If  an  alert  is  issued  for  a  marginal  event,  it  is  not  counted 
as  false  since  some  loss  with  shear  is  present. 

Since  a  dual-Doppler  analysis  is  available  approximately  every  60  seconds  and  algorithm  alerts 
are  issued  every  10  seconds,  linear  interpolation  between  dual-Doppler  alert  values  is  used  to  find 
the  dual-Doppler  alert  value  at  the  time  of  the  algorithm  alert.  We  required  the  time  difference  be¬ 
tween  the  dual-Doppler  analyses  just  before  and  after  the  algorithm  alert  time  be  less  than  90  se¬ 
conds.  If  the  difference  is  greater,  wind  shear  values  can  fluctuate  enough  to  make  linear  interpola¬ 
tion  questionable. 

4.2.  Alert  Statistics 

Alert  statistics  such  as  probability  of  detection  (POD)  and  probability  that  an  issued  alert  is  false 
(PFA)  are  computed  by  comparing  algorithm  alerts  to  a  set  of  comparison  alerts.  Each  operational 
runway  is  in  one  of  four  alert  states,  microburst  (MBA),  wind  shear  with  loss  (WSL),  wind  shear 
with  gain  (WSG),  and  no  alert  (Null).  The  alert  statistics  assess  the  ability  of  an  algorithm  to  place 
a  runway  in  the  same  alert  state  as  the  comparison  alert. 

Computing  alert  statistics  for  each  algorithm  consists  of  three  steps; 

1.  Build  contingent  )  tables. 

2.  Compute  detection  statistics  from  contingency  tables. 

3.  Compute  false  alert  statistics  from  contingency  tables. 

4.2.1.  Contingency  Tables 

Figure  2  shows  an  cxan'.plc  of  a  contingency  table.  Each  row  of  a  contingency  table  represents 
a  different  alert  state  as  detenr.  incd  by  the  algorithm:  MBA,  WSL,  WSG,  or  Null.  The  columns  rep¬ 
resent  the  same  alert  states  l<  t  the  comparison  alerts.  The  table  entries  are  filled  by  comparing  each 
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comparison  alert  state 

MBA  WSL  WSG  Null 

MBA 

algorithm 
alert  state 

Null 

Figure  2.  Contingency  table. 


algorithm  alert  to  its  associated  comparison  alert  and  incrementing  the  appropriate  entry.  After 
matching  all  of  the  alerts  the  table  contains  a  count  for  each  alert  pair.  These  counts  are  then  used 
to  compute  the  various  system  performance  probabilities. 

During  the  Doppler  processing,  the  data  are  smoothed  and  interpolated  to  the  grid  points  of  in¬ 
terest.  This  causes  errors  in  the  resulting  wind  field.  Additional  errors  in  the  comparison  alerts  are 
introduced  by  the  temporal  interpolation.  A  margin  of  error  of  ±5  knots  was  used  in  building  the 
contingency  table  to  account  for  these  inaccuracies. 

Figure  3  illustrates  the  effects  of  this  margin  of  error.  For  poll  1  only  a  Null  alert  is  ccmsidered 
correct.  For  polls  2  and  3  only  a  Null  alert  or  a  WSL  alert  is  considered  correct.  For  poll  4  only 
a  WSL  alert  is  considered  correct.  For  polls  5  and  6  only  a  WSL  alert  or  an  MBA  alert  is  COTsidered 
correct.  And  for  poll  7  only  an  MBA  alert  is  considered  correct.  For  example,  given  a  30-knot  algo¬ 
rithm  alert  and  the  27-knot  comparison  alert,  both  alerts  are  tallied  as  microburst  alerts.  Thus,  the 
counter  corresponding  to  the  first  row  and  first  colunm  would  be  incremented  by  one. 


-30  kts 


-15kts 


Dual -Doppler 
Alert+  Error  Bar 


T-T - 

4  5  6  7 
Time  (Polls) 


Figure  3.  Effect  of  the  5-knot  utKertainty  in  dual-Doppler  alerts. 
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4.2.2.  Computation  of  Detection  Statistics 


Three  measures  of  detection  were  used  to  evaluate  each  algorithm;  the  probability  of  a  loss  giv¬ 
en  a  microburst-POD(LIMB),  the  probability  of  a  loss  given  a  microburst  or  a  wind  shear  with  loss- 
POD(LIL),  and  the  probability  of  a  microburst  given  a  microburst-POD(MBlMB).  These  are  com¬ 
puted  from  the  contingency  table  built  using  comparison  alerts  from  the  narrow  runway  conidor. 

The  POD(LIMB)  is  the  probability  that  either  a  WSL  or  an  MBA  was  issued  when  the  compar  i¬ 
son  alert  indicates  an  MBA.  This  is  computed  from  the  contingency  table  by  taking  the  sum  of  the 
first  two  rows  of  the  first  column  and  dividing  by  the  total  of  all  the  elements  in  the  first  column. 

The  POD(LIL)  is  the  probability  that  either  a  WSL  or  an  MBA  was  issued  when  the  comparison 
alert  indicates  a  WSL  or  an  MBA.  This  is  computed  from  the  contingency  table  by  summing  the  four 
elements  in  the  first  two  rows  of  columns  one  and  two  and  dividing  by  the  total  of  all  the  elements 
in  the  first  two  columns. 

The  final  detection  statistic,  POD(MBIMB),  is  the  probability  that  an  MBA  was  issued  when 
the  comparison  alert  indicates  an  MBA.  This  is  computed  from  the  contingency  table  by  taking  the 
element  in  the  first  row,  first  column  and  dividing  by  the  total  of  all  the  elements  in  the  first  column. 

4.23.  Computation  of  False  Alert  Statistics 

Four  false  alert  statistics  were  used  to  evaluate  each  algorithm:  the  probability  of  false  micro¬ 
burst  (PFA(MB)),  the  probability  of  false  wind  shear  (PFA(WSL)),  the  probability  of  false  loss- 
PFA(L),  and  the  probability  of  microburst  over  warning  (OW(MB)).  These  are  computed  from  the 
contingency  table  built  using  comparison  alerts  from  the  wide  runway  corridor. 

The  PFA(MB)  is  the  probability  that  an  MBA  was  issued  when  the  comparison  alert  indicates 
no  loss.  This  is  computed  from  the  contingency  table  by  taking  the  sum  of  the  last  two  columns  of 
the  first  row  and  dividing  by  the  total  of  all  the  elements  in  the  first  row. 

The  PFA(WSL)  is  the  probability  that  a  WSL  alert  was  issued  when  the  comparison  alert  indi¬ 
cates  no  loss.  This  is  computed  from  the  contingency  table  by  taking  the  sum  of  the  last  two  columns 
of  the  second  row  and  dividing  by  the  total  of  all  the  elements  in  the  second  row. 

The  PFA(L)  is  the  probability  that  a  WSL  or  an  MBA  alert  was  issued  when  the  comparison 
alert  indicates  no  loss.  This  is  computed  from  the  contingency  table  by  summing  the  four  elements 
in  the  last  two  colunms  of  rows  one  and  two  and  dividing  by  the  total  of  all  the  elements  in  the  first 
two  rows. 

The  final  statistic,  OW(MB),  is  the  probability  that  an  MBA  alert  was  issued  when  the  compari¬ 
son  alert  indicates  a  WSL.  That  is,  the  alert,  while  not  false,  is  an  incorrect  use  of  MBA.  This  is  com¬ 
puted  from  the  contingency  table  by  taking  the  second  element  in  the  first  row  and  dividing  by  the 
total  of  all  the  elements  in  the  first  row. 
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4.3.  Loss  Accuracy  Histograms 

Another  important  measure  of  system  performance  is  the  ability  of  an  algorithm  to  correctly 
estimate  the  loss  associated  with  a  wind  shear.  This  is  evaluated  by  constructing  a  histogram  of  dif¬ 
ferences  between  algorithm  loss  estimates  and  comparison  alen  loss  estimates.  First  the  difference 
between  each  comparison  alert  value  and  corresponding  algorithm  aleit  value  is  computed.  Then 
the  histogram  is  built  by  dividing  the  range  of  differences  into  five-knot  bins  and  keeping  count  of 
the  number  of  differences  that  fall  into  each  bin. 

There  are  three  principal  characteristics  of  the  accuracy  histograms.  The  first  is  the  bias,  or  how 
closely  the  peak  of  the  histogram  coincides  with  the  center  bin  of  differences.  The  second  is  skew¬ 
ness,  or  how  symmetric  the  distribution  is.  Any  bias  or  skewness  in  the  histogram  indicates  a  tenden- 
■  cy  to  under  warn  or  over  warn.  The  third  is  variance,  or  how  much  the  accuracy  values  are  spread 
out  among  the  bins.  Ideally,  the  bin  values  should  cluster  strongly  around  the  central  bin. 

During  the  process  of  building  the  histograms  we  also  compute  the  fraction  of  loss  estimates 
that  are  within  5  knots,  10  knots,  etc.,  of  the  comparison  alert  loss  estimates. 
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5.  RESULTS 


5.1.  Alert  Statistics 

In  1991,  TDWR,  LLWAS  3,  and  four  integration  algorithms  were  evaluated  on  10  days.  Alerts 
were  generated  for  a  variety  of  runway  s,  both  parallel  and  non-parallel  to  the  TDWR  line  of  sight. 
TDWR,  and  thus  the  integration  algorithms,  are  expected  to  perform  best  on  runways  parallel  to  the 
TDWR  line  of  sight,  so  the  other  runways  are  included  to  test  the  algorithms  in  a  more  difficult  envi¬ 
ronment.  Results  are  reponed  for  the  parallel  runways,  non-parallel  runways,  and  all  the  runways 
as  a  group. 

The  real  runways  at  MCO  are  all  parallel  to  the  TDWR  line  of  sight  and  are  in  the  coverage 
region  of  the  LLWAS  sensors.  The  non-parallel  runways  have  LLWAS  coverage  provided  by  the 
Lincoln  anemometer  network.  The  data  quality  from  this  network  is  not  as  high  as  the  data  quality 
from  a  commissioned  LLWAS  3  network.  The  data  are  of  sufficient  quality  for  algorithm  compari¬ 
son  since  any  degradation  in  pierformance  will  be  experienced  by  each  integration  algorithm.  The 
LLWAS  3  performance  numbere  for  the  non-parallel  runways  should  be  considered  conservative 
estimates  of  the  true  performance  of  the  system. 

The  data  set  for  runways  that  are  parallel  to  the  TDWR  line  of  sight  is  much  more  extensive  than 
the  data  set  for  the  non-parallel  lunways  due  to  problems  with  the  Lincoln  anemometer  network. 
Only  6  percent  of  the  MB.*\  and  22  percent  of  the  WS/Loss  occur  on  non-parallel  runways.  This 
gives  greater  statistical  significance  to  the  performance  results  for  the  parallel  runways.  The  results 
for  the  non-parallel  runwa>  .s.  while  not  as  statistically  significant,  especially  for  measures  related 
to  MBA,  provide  insight  to  the  relative  benefits  of  the  different  integration  algorithms. 

5.1.1.  \99\  Without  Flight  Path  Shear  Integration 

The  TDWR  deployed  w  ith  LLWAS  3  systems  will  use  flight  path  shear  integration.  The  alert 
statistics,  shown  in  Tablc.s  3  through  5,  obtained  without  flight  path  shear  integration  are  included 
in  this  study  because  the  PL-A  algorithm  software  did  not  use  flight  path  shear  integration. 

The  detection  statistics  ax  very  high  for  each  algorithm.  The  probability  of  issuing  a  loss  alert 
when  a  microburst  strength  w  ind  shear  is  present  ai  the  flight  path  is  above  90  percent,  the  required 
minimum  detection  level.  .A  more  demanding  metric  is  the  probability  of  issuing  an  MBA  when  a 
microburst  strength  wind  shear  is  present  on  the  flight  path.  Each  algorithm  achieves  90  percent  or 
above  for  this  metric  as  well. 

The  maximum  allow  ed  PF  AfL)  is  10  percent  TDWR  without  flight  path  shear  integration  does 
not  meet  this  minimum.  h.iMng  a  PFA(L)  of  15  percent  overall,  and  38  percent  cai  those  runways 
that  are  not  parallel  to  the  TD  WR  l  i  ne  of  sight.  Despite  varying  success  at  removing  false  alerts,  some 
of  these  false  alerts  cany  thnnigh  the  integration  algorithms,  each  of  which  fails  to  achieve  the  10 
percent  maximum. 
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Both  the  PL-B  and  ML  algorithms  issue  fewer  false  microburst  alerts,  the  most  serious  of  the 
false  alerts,  than  the  Union  algorithm.  Of  the  four  integration  algorithms,  ML  has  the  lowest 
PFA(MB)  at  two  percent,  and  PL-A  has  the  highest  PFA(MB)  at  six  percent. 

ML.  with  a  PFA(WS)  of  19  percent,  is  the  only  integration  algorithm  with  a  lower  PFA(WS) 
than  the  Union  algorithm.  Of  the  four  integration  algorithms,  PL-A  has  the  highest  PFA(WS)  at 
23  percent. 

TDWR  without  flight  path  shear  integration  has  a  bias  towards  issuing  loss  alert  values  that  are 
greater  than  the  alert  values  determined  from  the  dual-Doppler  algorithm.  This  can  be  seen  from 
the  loss  accuracy  histograms  that  follow.  This  gives  rise  to  a  microburst  over  warning  (OW(MB)) 
of  3 1  percent  for  TDWR.  LLWAS  also  tends  to  over  warn.  This  is  due  primarily  to  the  compensation 
for  the  sparseness  of  the  anemometer  network,  not  a  high  bias.  The  microburst  over  warning  of  both 
TDWR  and  LLWAS  causes  the  integration  algorithms  to  over  warn  as  well.  Both  PL-B  and  ML  issue 
fewer  microburst  over  warnings  than  the  Union  algorithm.  Of  the  four  integration  algorithms,  ML 
has  the  lowest  OW(MB)  at  27  percent,  and  PL-A  has  the  highest  OW(MB)  at  37  percent. 

Table  3. 

1991  Probability  Statistics 
Without  Fight  Path  Shear  integration 
Runways  Parallel  to  TDWR  Line  of  Sight 


TDWR 

LLWAS 

UN 

PL-A 

PL-B 

POD(L|MB) 

99 

97 

99 

98 

100 

99 

POD(L|L) 

93 

77 

94 

93 

94 

93 

POD(MBIMB) 

97 

91 

98 

97 

99 

97 

PFA(MB) 

3 

2 

4 

4 

2 

1 

PFA{WS) 

11 

1 

11 

13 

12 

10 

PFA(L) 

7 

2 

7 

8 

7 

6 

OW(MB) 

28 

22 

30 

32 

29 

24 

Table  4. 

1991  Probability  Statistics 

Without  Flight  Path  Shear  Integration 

Runways  Not  Parallel  to  TDWR  Line  of  Sight 

TDWR 

LLWAS 

UN 

PL-A 

OL-B 

ML 

POD(LIMB) 

100 

92 

100 

100 

100 

100 

POD(L|L) 

87 

71 

91 

93 

91 

91 

POD(MB|MB) 

98 

85 

98 

99 

99 

98 

PFA(MB) 

17 

10 

14 

16 

13 

11 

PFA(WS) 

43 

3 

43 

42 

41 

36 

PFA(L) 

38 

6 

36 

33 

36 

32 

OW(MB) 

51 

51 

57 

59 

53 

52 

20 


Table  5. 

1991  Probability  Statistics 
Without  Flight  Path  Shear  Integration 
Ail  Runways 


TDWR 

LLWAS 

UN 

PL-A 

PL-B 

ML 

POD(L|MB) 

99 

97 

99 

98 

100 

99 

POD{L|L) 

92 

76 

93 

93 

94 

93 

POD(MB|MB) 

97 

90 

98 

97 

99 

97 

PFA(MB) 

4 

3 

5 

6 

4 

2 

PFA(WS) 

22 

2 

22 

23 

22 

19 

PFA(L) 

15 

2 

15 

14 

15 

13 

OW(MB) 

31 

25 

33 

37 

31 

27 

5.1.2.  1991  With  Flight  Path  Shear  Integration 

The  alert  statistics  in  this  section,  shown  in  Tables  6  through  8,  more  accurately  reflect  the  ex¬ 
pected  performance  of  a  fielded  system  since  all  TDWR  co-located  with  LLWAS  3  will  use  flight 
path  shear  integration.  The  PL-A  algorithm  is  not  evaluated  in  this  section  since  it  did  not  use  TDWR 
with  flight  path  shear  integration. 

All  of  the  algorithms  detect  wind  shear  with  a  loss  of  head  wind  very  well.  For  each  algorithm, 
the  probability  of  issuing  a  loss  alert  when  a  microburst  strength  wind  shear  is  present  on  the  flight 
path  is  above  90  percent,  the  required  minimum  detection  level.  A  more  demanding  metric  is  the 
probability  of  issuing  an  MBA  when  a  microburst  strength  wind  shear  is  present  on  the  flight  path. 
Each  algorithm  achieves  90  percent  or  above  for  this  metric  as  well. 

The  false  alert  statistics  show  a  large  improvement  for  TDWR  and  the  integration  algorithms 
when  flight  path  shear  integration  is  used.  The  maximum  allowed  PFA(L)  is  10  percent.  All  of  the 
algorithms  meet  this  standard.  TDWR  still  issues  an  excessive  number  of  false  alerts  on  rurways 
that  are  not  parallel  to  the  TDWR  line  of  sight.  The  integration  algorithms  give  a  slight  reduction 
in  the  number  of  false  alerts;  however,  their  PFA(L)  remains  high  on  these  runways. 

Both  the  PD-B  and  ML  algorithms  issue  fewer  false  microburst  alerts,  the  most  serious  of  the 
false  alerts,  than  the  Union  algorithm.  Of  the  three  integration  algorithms,  ML  has  the  lowest 
PFA(MB)  at  one  percent,  and  the  Union  has  the  highest  PFA(MB)  at  three  percent.  ML  also  has  the 
lowest  PFA(WS)  at  nine  percent,  and  the  Union  has  the  highest  PFA(WS)  at  1 1  percent. 

TDWR  with  flight  path  shear  integration  achieves  a  much  lower  OW(MB)  than  without  flight 
path  shear  integration.  The  integration  algorithms  still  must  contend  with  the  microburst  over  warn¬ 
ing  from  LLWAS.  Both  PD-B  and  ML  reduce  microburst  over  warning  relative  to  the  Union  algo¬ 
rithm,  with  ML  giving  the  greatest  reduction.  Of  the  three  integration  algorithms,  ML  has  the  lowest 
OW(MB)  at  14  percent,  and  Union  has  the  highest  OW(MB)  at  23  percent. 
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Table  6. 

1991  Probability  Statistics 
With  Flight  Path  Shear  Integration 
Runways  Parallel  to  TDWR  Line  of  Sight 


TDWR  LLWAS _ UN  PL-B _ MJ 


POD(L|MB) 

98 

97 

99 

99 

98 

POD(L|L) 

90 

77 

91 

92 

90 

POD(MB|MB) 

96 

91 

97 

98 

96 

PFA(MB) 

0 

2 

2 

1 

0 

PFA(WS) 

1 

1 

1 

1 

2 

PFA(L) 

1 

2 

2 

1 

1 

OW(MB) 

5 

22 

20 

14 

12 

Table  7. 

1991  Probability  Statistics 
With  Flight  Path  Shear  Integration 
Runways  Not  Parallel  to  TDWR  Line  of  Sight 


TDWR 

LLWAS 

UN 

PL-B 

ML 

POD(L|MB) 

100 

92 

100 

100 

100 

POD(LIL) 

82 

71 

90 

90 

88 

POD(MB|MB) 

91 

85 

92 

91 

91 

PFA(MB) 

13 

10 

11 

13 

12 

PFA(WS) 

31 

3 

30 

28 

26 

PFA(L) 

29 

6 

26 

26 

24 

OW(MB) 

37 

51 

51 

46 

41 

Table  8. 

1991  Probability  Statistics 
With  Flight  Path  Shear  Integration 
All  Runways 


TDWR 

LLWAS  _ 

UN 

PL-B 

ML 

POD(L|MB) 

98 

97 

99 

100 

98 

POD{LlL) 

89 

76 

91 

92 

90 

POD(MB|MB) 

96 

90 

97 

98 

96 

PFA(MB) 

1 

3 

3 

2 

1 

PFA(WS) 

10 

2 

11 

10 

9 

PFA(L) 

7 

2 

8 

7 

7 

OW(MB) 

8 

25 

23 

17 

14 

22 


5.1.3.  1992  With  Flight  Path  Shear  Integration 

In  1992  only  TDWR  with  flight  path  shear  integration,  LLWAS  3,  and  the  ML  algorithm  were 
tested.  Due  to  poor  quality  Doppler  data  from  the  UND  radar,  only  runways  oriented  north/south 
were  used  in  the  evaluation.  The  alen  statistics  are  shown  in  Table  9. 

The  detection  statistics  are  slightly  below  those  achieved  in  1991.  Both  TDWR  and  Message 
Level  achieve  a  POD(MB)  well  above  90  percent.  LLWAS  has  a  POD(MB)  of  87  percent,  slightly 
below  the  required  90  percent.  The  LLWAS  anemometer  network  does  not  extend  to  the  full  three 
miles  from  the  runways,  making  it  very  difficult  to  achieve  a  90  percent  detection  probability  over 
the  entire  airport  region. 

The  false  alert  statistics  are  higher  than  in  1991.  However,  the  PFA(L)  values  are  well  below 
the  maximum  allowed  value  of  10  percent,  and  each  algorithm  issues  very  few  false  microburst 
alerts.  The  microburst  over  warning  statistics  are  also  higher  than  in  1991. 

Table  9. 

1992  Probability  Statistics 
With  Flight  Path  Shear  Integration 
Runways  Parallel  to  TDWR  Line  of  Sight 


TDWR 

LLWAS 

ML 

POD(LIMB) 

96 

87 

97 

POD(L|L) 

88 

71 

86 

POD(MB|MB) 

86 

66 

86 

PFA(MB) 

2 

6 

4 

PFA(WS) 

3 

3 

3 

PFA(L) 

2 

4 

4 

OW(MB) 

17 

28 

21 

5.2.  Accuracy  Statistics 

Results  on  the  accuracy  of  the  loss  estimates  are  reported  in  two  forms.  Both  relate  to  the  accura¬ 
cy  of  the  loss  estimates  for  times  and  runways  where  the  comparison  alerts  indicate  a  microburst 
level  hazard.  The  first  form  is  a  table  giving  the  percentage  of  algorithm  alerts  with  a  loss  estimate 
within  a  fixed  number  of  knots  of  the  comparison  alert  loss  estimate,  and  the  median  error  in  knots. 
This  gives  an  overall  assessment  of  system  accuracy  and  bias.  The  second  form  are  histograms  giv¬ 
ing  the  percentage  of  alerts  that  differ  from  the  comparison  alert  values  by  a  multiple  of  5  knots  ±2.5 
knots. 

5.2.1.  1991  Without  Flight  Path  Shear  Integration 

Table  10  gives  the  percentage  of  alerts  that  fall  within  a  range  of  the  loss  estimate  from  its 
associated  comparison  alert  and  the  median  error  in  knots.  The  table  shows  LLWAS  is  more  accurate 
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than  TDWR  without  flight  path  shear  integration.  The  integration  algorithms  are  slightly  more  accu¬ 
rate  than  TDWR,  but  none  are  as  accurate  as  LLWAS.  The  ML  algorithm  is  significantly  more  accu¬ 
rate  than  the  other  integration  algorithms. 


Table  10. 

1991  Accuracy  Statistics 
Without  Flight  Path  Shear  integration 


TDWR 

LLWAS 

Union 

PL-A 

PL-B 

%  +  2.5  knots 

8 

16 

8 

8 

8 

17 

%  +  7.5  knots 

28 

50 

28 

31 

28 

43 

%  +  12.5  knots 

53 

74 

54 

59 

54 

63 

%±  17.5  knots 

70 

88 

71 

76 

71 

78 

median  error  (kts) 

11.7 

3.5 

11.6 

10.7 

11.6 

8.6 

Figure  4  shows  that  TDWR  without  flight  path  shear  integration  consistently  over  estimates  the 
strength  of  the  loss  associated  with  microburst  level  events,  being  centered  at  1 1.7  knots.  This  is  the 
expected  result;  complaints  of  microburst  overwaming  led  to  the  development  of  the  flight  path 
shear  integration  algorithm  to  refine  the  TDWR  loss  estimates. 

Figure  5  shows  that  LLWAS  on  average  produces  accurate  loss  estimates,  with  a  histogram  cen¬ 
tered  at  3.5  knots.  The  wide  spread  of  values  is  the  result  of  the  undersampling  of  the  microburst 
outflow  due  to  the  sparseness  of  the  anemometer  network,  and  the  corresponding  statistical  correc- 
ticMi  for  the  undersampling.  The  LLWAS  results  are  independent  of  flight  path  shear  integration. 

Figures  6  and  7  for  the  Union  and  PL-B  integration  algorithms  are  very  similar  to  the  histogram 
for  TDWR.  The  fact  that  these  algorithms  do  not  produce  a  significant  increase  in  the  over  estimation 
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Figure  4.  1991  TDWR  loss  accuracy  histogram  without  flight  path  shear  integration. 
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Figure  5.  1991  LLWAS  loss  accuracy  histogram. 


alert  value  -  comparison  value  (knots) 


Figure  6.  1991  Union  (PL-B)  loss  accuracy  histogram  without  flight  path  shear  integration. 


of  the  loss  values  relative  to  TDWR  arises  because  the  TDWR  alerts  are  nearly  always  stronger  than 
the  LLWAS  alerts  so  the  integrated  alert  values  are  usually  the  TDWR  alert  values.  Figure  8  shows 
that  the  averaging  of  matched  loss  estimates  by  the  ML  algorithm  has  reduced  the  amount  of  the  over 
estimatioi  of  the  loss  value,  however  a  significant  amount  remains. 

5.2,2.  1991  With  Flight  Path  Shear  Integration 

The  TDWR  flight  path  shear  integration  algorithm  was  developed  to  reduce  the  over  estimation 
in  the  TDWR  loss  estimates.  Figure  9  and  table  11  show  that  with  this  algorithm  TDWR  produces 
more  accurate  loss  estimates  without  incurring  a  significant  amount  of  underwaming.  TDWR  issues 
loss  estimates  that  arc  closer  to  the  comparison  loss  estimates  than  LLWAS,  although  TDWR  has 
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Figure  7.  1991  PL-A  loss  accuracy  histogram  without  flight  path  shear  integration. 
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Figure  8.  1991  ML  loss  accuracy  histogram  without  flight  path  shear  integration. 


a  larger  median  error  than  LLWAS.  This  apparent  discrepancy  arises  from  the  larger  spread  of  the 
errors  in  the  LLWAS  loss  estimates  relative  to  the  spread  of  errors  in  the  TDWR  loss  estimates,  as 
seen  in  figures  5  and  9.  The  percentage  of  loss  estimates  from  the  Union  andPL-B  algorithms  near 
the  loss  estimates  from  the  comparison  alerts  are  similar  to  those  for  LLWAS,  although  their  median 
errors  are  larger.  The  ML  algonthm  has  the  highest  accuracy  of  the  integration  algorithms  and  pro¬ 
duces  more  accurate  loss  estimates  than  TDWR  and  LLWAS  by  most  measures. 
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Figure  9.  1991  TDWR  loss  accuracy  histogram  with  flight  path  shear  integration. 


Table  11. 

1991  Accuracy  Statistics 
With  Flight  Path  Shear  Integration 


TDWR 

LLWAS 

PL-B 

%  +  2.5  knots 

19 

16 

15 

16 

23 

%  +  7.5  knots 

57 

50 

47 

49 

60 

%  +  12.5  knots 

83 

74 

75 

76 

81 

%  ±  17.5  knots 

93 

88 

89 

90 

93 

median  error  (kts) 

5.6 

3.5 

7.1 

4.6 

Figure  9  shows  that  TDWR  with  flight  path  shear  integration  has  a  high  bias,  but  this  bias  is 
less  than  before  flight  path  shear  integration  was  added.  The  histogram  is  nearly  symmetric,  and  cen¬ 
tered  at  5.6  knots.  The  results  in  figure  5  still  hold  for  LLWAS,  since  LLWAS  is  not  affected  by  the 
introduction  of  flight  path  shear  integration.  Figures  10  and  1 1  show  that  both  the  Union  and  PL-B 
algorithms  have  a  higher  bias  than  either  TDWR  or  LLWAS.  Their  histograms  are  nearly  symmetric, 
centered  at  7.5  knots  and  7.1  knots,  respectively,  and  have  a  spread  similar  to  the  histogram  for 
TDWR.  Figure  12  shows  that  the  ML  algorithm  has  a  high  bias,  although  it  is  reduced  from  both 
TDWR  and  the  other  integration  algorithms  and  approaches  the  bias  in  LLWAS.  The  ML  histogram 
is  nearly  symmetric  and  has  a  spread  similar  to  the  histogram  for  TDWR. 

5.23.  1992  With  Flight  Path  Shear  Integration 

The  accuracy  statistics  for  1992  indicate  that  the  performance  of  the  systems  was  not  as  good 
as  in  1991.  Table  12  gives  the  accuracy  statistics  for  TDWR  with  flight  path  shear  integratitm. 
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alert  value  -  comparison  value  (knots) 

Figure  10.  1991  Union  TDWR  loss  accuracy  histogram  with  flight  path  shear  integration. 


alert  value  -  comparison  value  (knots) 


Figure  11.  1991  PL-B  loss  accuracy  histogram  with  flight  path  shear  integration. 


LLWAS  3,  and  the  ML  algorithm.  Unlike  in  1991,  LLWAS  produced  more  accurate  loss  estimates 
than  TDWR  as  measured  by  both  distributicm  of  errors  and  median  error.  The  performance  of  the 
ML  algorithm  is  between  that  of  LLWAS  and  TDWR. 
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Figure  12.  1991  ML  loss  accuracy  histogram  with  flight  path  shear  integration. 


Table  12. 

1991  Accuracy  Statistics 
With  Flight  Path  Shear  Integration 


TDWR 

LLWAS 

ML 

%  +  2.5  knots 

15 

18 

15 

%  +  7.5  knots 

43 

47 

45 

%  + 12.5  knots 

68 

70 

70 

%+ 17.5  knots 

84 

87 

86 

median  error  (kts) 

7.6 

5.3 

7.2 

Figure  13  shows  a  higher  bias  in  TDWR  than  in  1991.  The  histogram  is  centered  at  7.6  knots 
and  is  fairly  symmetric.  Figure  14  shows  that  LLWAS  in  1992  also  had  a  higher  bias  than  in  1991, 
although  the  spread  of  values  is  reduced.  The  LLWAS  histogram  is  centered  at  5.3  knots,  giving  a 
lower  bias  than  TDWR.  Figure  15  shows  that  the  ML  algorithm  has  a  smaller  spread  than  either 
TDWR  or  LLWAS  and  is  centered  at  7.2  knots. 
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Figure  13.  1992  TDWR  loss  accuracy  histogram  with  flight  path  shear  megration. 


alert  value  -  comparison  value  (knots) 


Figure  14.  1991 LLWAS  loss  accuracy  histogram. 
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Figure  15.  1992  ML  loss  accuracy  histogram  with  flight  path  shear  integration. 
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6.  CONCLUDING  REMARKS 


Each  of  the  algorithms  performed  very  well.  With  the  introduction  of  the  flight  path  shear  in¬ 
tegration  to  TDWR,  all  of  the  integration  algorithms  easily  bettered  the  required  90  percent  probabil¬ 
ity  of  detection  for  microburst  level  wind  shear  events,  with  less  than  the  10  percent  maximum  al¬ 
lowed  false  alert  percentage  when  considered  over  all  runways  studied.  All  of  the  algorithms  have 
a  high  bias,  but  at  levels  that  are  not  detrimental  to  airport  operations.  By  each  measure  studied,  the 
ML  integration  algorithm  provided  performance  superior  to  the  other  integration  algorithms. 

Based  on  an  extensive  review  of  the  algorithms,  evaluation  methodology,  and  results,  NCAR 
and  Lincoln  Laboratory  issued  a  joint  recommendation  to  the  FAA  that  the  ML  algorithm  be  chosen 
as  the  production  TDWR/LLWAS  3  integration  algorithm.  Raytheon  is  incorporating  this  algorithm 
into  build  5  of  the  TDWR  software. 

The  excellent  performance  reported  for  TDWR  and  LLWAS  3  is  due  in  part  to  the  test  location. 
The  Orlando  environment  is  particularly  favorable  to  wind  shear  detection  algorithms.  Microbursts 
there  are  usually  large,  symmetric,  and  have  a  high  moisture  content  and  so  are  easier  for  the  integra¬ 
tion  subsystems  todetect.  However,  even  in  a  benign  environment,  integration  has  an  advantage  over 
LLWAS  3  in  detection  of  wind  shear  with  a  loss  of  head  wind  due  to  TDWR’s  greater  coverage  re¬ 
gion  and  spatial  density  of  data.  Integration  also  has  an  advantage  over  TDWR  due  to  LLWAS  3’s 
rapid  update  rate  and  superior  ability  to  detect  wind  shear  with  a  gain  of  head  wind,  which  was  not 
considered  in  this  study. 

In  1992,  NCAR  conducted  an  operational  demonstration  of  the  ML  algorithm  at  Stapleton  In¬ 
ternational  Airport.  The  NCAR  results  in  Denver  show  the  benefits  of  integrating  TDWR  and 
LLWAS  3  are  much  greater  in  that  environment  than  in  Orlando  [7]. 
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ACRONYMS  AND  ABBREVIATIONS 


ARTCC 

Air  Route  Traffic  Control  Center 

FAA 

Federal  Aviation  Administration 

GMT 

Greenwich  Mean  Time 

kt 

knot 

LLWAS  3 

Low  Level  Wind  Shear  Alert  System,  third  generation 

MB 

Microburst 

MBA 

Microburet  Alert 

MCO 

Orlando  International  Airport 

MIT/LL 

MIT  Lincoln  Laboratory 

ML 

Message  Level  algorithm 

NCAR/RAP 

National  Center  for  Atmospheric  Research/Research  Applications  group 

Null 

No  alert 

OW 

Over  Warning 

PFA 

Probability  of  False  Alen 

PFA(L) 

Probability  of  False  Alert  (Loss) 

PFA(MB) 

Probability  of  False  Alert  (Microburst) 

PFA(WS) 

Probability  of  False  Alert  (wind  shear) 

PIREP 

Pilot  Report 

PL-A 

Product  Level-A  algorithm 

PL-B 

Product  Level-B  algorithm 

POD 

Probability  of  Detection 

TDWR 

Terminal  Doppler  Weather  Radar 

TRACON 

Terminal  Radar  Approach  Control 

UND 

University  of  North  Dakota 

WSA 

Wind  Shear  Alert 

WSL 

Wind  Shear  with  Loss 

WSG 

Wind  Shear  with  Gain 
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APPENDIX  A 


A  PIREP.BASED  ANALYSIS  OF  AN  INTEGRATED 
TDWR/LLWAS  ALERT  SYSTEM 
EVALUATED  AT  ORLANDO  INTERNATIONAL  AIRPORT 
DURING  THE  SUMMER  OF  1992 

Lloyd  Stevenson 

John  A.  Volpe  Nation^  Transportation  Systems  Center 


During  the  summer  of  1992,  the  Terminal  Doppler  Weather  Radar  (TD^iVR)  Program 
evaluated  a  system  that  provided  wind  shear  and  microburst  alerts  to  pilots  landing  and  departing 
from  Orlando  International  Airport.  The  demonstration  took  place  over  the  65-day  period 
starting  May  4  and  ending  July  7.  For  a  majority  of  the  alert  periods,  tapes  of  the  pilot/Local 
Control  radio  communication  channels  were  obtained  for  analysis.  The  tapes  provided  a 
portrayal  of  airpon  runway  operations  during  the  alert  periods,  including  any  pilot  reports 
(PIREPs)  of  weather-related  encounters. 

This  appendix  presents  the  results  of  an  operational  analysis  of  the  radio  communication 
tapes  and  other  materials,  and  represents  one  component  of  the  overall  evaluation.  The  study 
pursued  two  lines  of  investigation:  (1)  what  happened  operationally  during  the  alert  periods,  and 
(2)  what  would  have  been  the  operational  impact  of  alternative  alert  schemes? 

Relative  to  “what  happened  operationally,”  the  primary  findings  are: 

•  The  tested  system  performed  well  overall,  based  on  pilot  reports,  and 

•  There  were  two  surprises  resulting  in  recommendations  concerning  the  issuance 
of  alerts  by  controllers. 

Relative  to  “the  operational  impact  of  alternative  alert  schemes,”  the  primary  frndings  are: 

•  The  post-1990  software  changes  significantly  improved  the  quality  of  the  alert 
service  provided  by  TDWR,  and 

•  System  integration  provides  a  superior  overall  alert  service  over  what  would  be 
provided  by  either  TDWR  or  the  Phase-3  Low  Level  Wind  Shear  Alert  System 
(LLWAS)  as  stand-alone  alert  systems. 

The  U.S.  Department  of  Transportation,  Federal  Aviation  Administration,  ANR-150, 
sponsored  the  work  presented  in  this  appendix.  The  Volpe  National  Transportation  Systems 
Center  of  the  U.S.  Department  of  Transportation  conducted  the  operational  analysis.  The 
Massachusetts  Institute  of  Technology’s  Lincoln  Laboratory  provided  technical  support. 

BACKGROUND 

Wind  shear  in  general  and  microbursts  in  particular  are  of  concern  to  landing  and  departing 
pilots.  One  FA  A  activity  addressing  this  concern  is  the  development  of  TDWR. 

The  1992  demonstration  was  one  of  a  series  of  such  demonstrations  conducted  by  the 
TDWR  Program  since  1988.  The  series  has  served  as  a  vehicle  for  perfecting  the  basic  TDWR 
system  and  associated  weather  products  and  for  investigating  advanced  TD\^-related  system 
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concepts  and  services.  The  1992  demonstration  involved  an  Integrated  TDWR/LLWAS  Alert 
System  consisting  of  functional  prototypes  of  both  TDWR  and  LLWAS-3. 

Alert  terminology — In  issuing  an  alert  to  a  landing/departing  pilot,  the  procedure  called  for 
Local  Control  to  start  by  identifying  it  as  either  a  “wind  shear  alert”  or  a  “microburst  alert.” 
Wind  shear  alerts  were  of  two  kinds:  (1)  “gain”  alerts  (e.g.,  20-kt  gain  2-Mile  Final),  which  were 
used  to  identify  areas  of  increasing  head  winds  associated  with  such  wind  shear  features  as  gust 
fronts,  and  (2)  “loss”  alerts  (e.g.,  20-kt  loss  2-Mile  Final),  which  were  used  to  identify  areas  of 
decreasing  head  winds  associated  with  weak  microbursts.  Loss  alerts  for  intensities  of  30  knots 
or  more  were  identified  as  microburst  alerts  to  highlight  the  greater  potential  hazard. 

SUMMARY  AND  CONCLUSIONS 

1.  The  analysis  covered  at  least  79  percent  of  the  *‘dlert**  wind  shears  that  occurred 
during  the  demonstration.  The  on-site  demonstration  team  logged  130  wind  shear 
features  as  possibly  causing  alerts  to  be  issued  to  pilots  landing  and  departing  Orlando 
International  Airport  during  the  65-day  demonstration.  The  pilot-controller  radio 
communication  tapes  were  analyzed  for  103  (or  79  percent)  oTthe  130  wind  shear 
features,  which  resulted  in  the  analysis  of  39  alert  periods  lasting  a  total  of  10  hours. 

2.  Pilots  received  alerts  from  two  systems  during  the  demonstration:  (1)  TDWR-based 
alerts  from  May  4  though  June  19,  while  the  Integrated  TDWR/LLWAS  Alert  Software 
Package  imderwent  its  pre-demonstration  checkout,  and  (2)  integrated  alerts  from  June  20 
through  the  end  of  the  demonstration  on  July  7.  TTie  39  examined  alert  periods  includes 
periods  when  pilots  received  TDWR-bas^  alerts  and  periods  when  pilots  received 
integrated  alerts. 

3.  Both  alert  systems  used  during  the  demonstration  performed  well  in  providing  timely 
alerts  of  si^ficant  wind  shear  conditions.  The  105  PIREPs  received  from  lantSng  and 
departing  pilots  during  the  39  examined  alert  periods  support  this  finding. 

4.  An  examination  of  the  alerts  issued  by  Local  Control  identified  two  innovations  being 
used  by  at  least  some  of  the  controllers  and  one  potential  problem  area. 

Innovation  one:  **adjacent'airspace-alerP*  advisories — One  third  of  the  alerts  issued  by 
Local  Control  did  not  affect  the  air  crews  directly  but  advised  them  of  an  alert  in  effect  for 
some  adjacent  airspace.  These  situations  primarily  involved  landing  air  crews  being  advised 
of  an  alert  in  effect  for  the  airspace  off  the  departure  end  of  the  arrival  runway. 

RECOMMENDATION — ^The  apparent  use  of  “adjacent-airspace-alert”  advisories  as 
heads-up  information  for  landing  pilots  in  the  event  of  a  missed  approach  should  be 
examined  for  possible  inclusion  into  the  formal  guidelines  to  controllers  on  the 
issuance  of  alerts  that  will  accompany  the  introduction  of  the  operational  system. 
Perhaps,  these  extra  alerts  could  be  issued  on  a  “time-permitting”  basis. 

Innovation  two:  use  of  non-standard  phraseology — Controllers  used  non-standard 
phraseology  in  issuing  three  quarters  of  those  alerts  that  located  wind  shear  on  the  runway. 
Instead  of  using  the  standard  phraseology  of  “over  the  runway”  in  those  cases,  the  controllers 
substituted  in  the  phrase  “approach  end  of  the  runway”  or  “departure  end  of  the  runway”  in 
an  apparent  attempt  to  provide  more  detailed  location  information. 

Potential  problem  area:  “non-issued”  alerts — ^Local  Control  did  not  issue  22  percent  of 
the  alerts  called  for  during  the  39  examined  alert  periods.  These  situations  occurred  at  alert 
startup  when  Local  Control  did  not  issue  the  alert  in  effect  to  the  first  one  or  more  air  crews. 
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RECOMMENDATION — This  number  of  “non-issued”  alerts  may  warrant  a  re¬ 
examination  of  the  means  used  to  get  controller  anention  at  alen  startup,  a  problem 
area  identified  in  earlier  demonstrations. 

5.  Trend  is  for  increasing  pilot  utilization  of  alerts  for  microburst  avoidance-Relative  to 
earlier  demonstrations,  more  pilots  used  the  issued  alerts  to  avoid  microburst  encounters. 
For  example,  in  the  19W  TDWR  Demonstration  at  Orlando,  a  pilot  landed/departed  with 
a  microburst  alert  in  effect,  on  average,  once  every  one  to  two  days  over  the 
demonstration  period.  This  average  was  reduced  to  one  pilot  every  ten  days  during  the 
1992  Demonstration,  representing  a  significant  improvement  in  microburst  avoidance. 

6.  One  pilot  that  landed  with  a  microburst  alert  in  effect  experienced  difficulty — ^The  pilot, 
who  had  been  issued  a  30-knot  microburst  alert,  ^vised  Local  Control  after  landing  that 
nobody  else  should  land  to  which  the  controller  observed:  “You  sure  beat  that  one  by  a 

whisker . just  as  you  got  to  the  other  side  of  it,  we  had  a  microburst  alert  with  a 

60-knot  loss  on  3-Mile  Final.” 

7.  The  following  are  the  results  of  a  “what  if’  comparison  of  four  alert  systems/softwares 
relative  to  what  their  operational  impact  would  have  been  over  the  39  examined  alert 
periods: 

a.  TDWR  with  1990  software  would  have  maximized  alert  activity — TDWR  with 
1990  software,  as  a  stand-alone  alert  system,  would  have  maximized  the  alert 
activity  during  the  demonstration  with  13.5  hours  of  overall  alert  period  duration 
and  233  air  crews  being  issued  an  alert. 

b.  The  post-1990  software  changes  safely  and  significantly  reduced  TDWR  alert 
activity— -Vm  post- 1990  TDV^  software  changes  significantly  reduced  TDWR 
alen  activity  by:  (1)  21  percent  in  terms  of  overall  alert  duration  during  the 
demonstration  [from  13.5  to  10.7  hours],  (2)  36  percent  in  terms  of  the  numl^  of 
air  crews  issued  an  alert  [from  233  to  148  air  crews],  and  (3)  68  percent  in  terms 
of  the  number  of  air  crews  issued  a  microburst  alert  [from  ^  to  21  air  crews].  An 
examination  of  the  associated  PIREPs  suggests  that  the  reduction  in  alert  activity 
was  justified. 

c.  LLWAS  would  have  minimized  alert  activity — ^The  LLWAS  channel  of  the 
inte^ted  TDWR/LLWAS  system,  as  a  stand-alone  alert  system,  would  have 
minimized  the  alert  activity  during  the  demonstration  with  5.7  hours  of  overall 
alert  period  duration  and  87  air  crews  being  issued  an  alert.  It  should  be  noted 
that  LLWAS  alert  coverage  did  not  start  until  2-Mile  Final  during  the 
demonstration  versus  3-Mile  Final  for  TDWR,  which,  in  part,  explains  the  fewer 
alerts. 

d.  TDWR/LLWAS  integration  into  a  single  alert  system  would  have  provided  a 
superior  overall  alert  service — ^The  intention  of  TDWR  and  LLWAS  into  a 
single  alert  system  has  the  potential  to  increase  alert  coverage,  improve  alert 
timeliness,  decrease  extraneous  alerts,  and  improve  alert  accuracy  relative  to  what 
either  system  can  do  alone.  The  demonstration  provided  examples  in  each  of 
these  areas.  With  integration,  alert  activity  during  the  39  examined  alert  periods 
would  have  been  10.5  hours  in  overall  alert  period  duration  and  141  air  crews 
would  have  been  issued  an  alert.  This  represents  a  slight  decrease  in  alert  activity 
over  the  TDWR  channel  alone  and  a  sharp  increase  over  the  alert  activity 
exhibited  by  the  LLWAS  channel  alone. 
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8.  Generalizing  these  operational  results  from  Orlando  to  other  airports  is  complicated — 
Generalizing  these  results  to  other  airports  is  complicated  by  the  fact  that  the  Orlando 
wind  shear  environment  represents  a  relatively  benign  versus  a  worst-case  demonstration 
site  from  an  operational  viewpoint.  Relative  to  other  airports,  the  Orlando,  summer,  wind 
shear  environment  is  characterized  by:  (1)  few  gust  fironts,  and  they  tend  to  be  weak;  and 
(2)  numerous  microbursts,  but  the  “wet”  atmospheric  conditions  at  Orlando  allow  pilots 
and  controllers  to  associate/locate  microbursts  with  rain  shafts  at  the  surface  versus 
having  to  deal  with  them  in  “clear  air.” 

ANALYSIS  COVERED  AT  LEAST  79  PERCENT  OF  THE  “ALERT”  WIND  SHEARS 
THAT  OCCURRED  DURING  THE  DEMONSTRATION 

The  on-site  demonstration  team  maintained  a  log  of  all  observed  wind  shear  features  that  may 
have  generated  alerts  at  Orlando  International  Airport  together  with  the  maximum  observed 
change  in  horizontal  wind  speed  with  distance  observed  for  each  of  the  features.  Of  the  130 
wind  shear  features  logged  over  the  nine- week  demonstration,  79  percent  [or  103]  have  been 
included  in  the  analysis.  Specifically,  the  analysis  includes:  (1)  91  percent  [or  86]  of  the  94 
logged  microbursts,  which  ranged  in  their  maximum  observed  intensities  from  20  to  70  knots, 
and  (2)  47  percent  [or  17]  of  the  36  logged  gust  fronts  and  sea  breezes,  which  ranged  in  their 
maximum  observed  intensities  from  15  to  25  ^ots. 

The  103  wind  shear  events  included  in  the  analysis  resulted  in  39  alert  periods  that  lasted  a 
total  of  612  minutes  or  slightly  over  10  hours.  The  alert  periods  ranged  in  duration  from  less 
than  1  minute  up  to  38  minutes  and  averaged  15  minutes. 

There  is  no  formal  definition  as  to  what  constitutes  an  alert  period.  For  analysis  purposes,  an 
alert  period  was  defined  to  start  with  the  first  alert  generated  for  the  airport  and  to  end  whenever 
alerts  for  the  airport  ceased  to  be  generated  for  a  period  in  excess  of  five  minutes.  Consequently, 
a  startup  of  alerts  in  less  than  six  minutes  after  they  had  ceased  was  considered  to  be  a 
continuation  of  the  original  alert  period. 

WHAT  HAPPENED  OPERATIONALLY  DURING  THE  39  EXAMINED  ALERT 
PERIODS 

This  section  is  concerned  with  the  alerts  issued  to  pilots  by  Local  Control  without  taking  into 
account  their  source  (i.e.,  whether  they  were  TDWR-based  or  integrated  alerts).  An  analysis  of 
the  generated  alerts  by  source  is  presented  in  the  next  section. 

Runway  Operations  Ceased  Many  Times  and  for  a  Variety  of  Reasons 

Either  arrival  runway  operations  or  departure  runway  operations  or  both  operations  ceased 
for  at  least  5  minutes  during  21  of  the  39  alert  periods  examined.  Operations  ceased  23  times 
during  these  21  alert  periods  for  periods  lasting  up  to  30  minutes  and  averaging  15  minutes  in 
duration.  Overall  the  operations  ceased  for  6  of  the  10  hours  that  alerts  were  in  effect.  The 
communication  tapes  permitted  at  least  some  of  the  factors  influencing  the  cessation  of  runway 
operations  to  be  identified  in  each  of  the  23  cases. 

8  of  the  23  periods  did  not  involve  microburst  alerts  but  involved  such  factors  as:  (1) 
pilot  decisions  not  to  land  or  take  off  due  to  the  appearance  of  the  weather  in  Ae 
vicinity  of  the  ai^ort  and/or  wind  shear  alerts  being  in  effect,  or  (2)  an  Air  Traffic 
management  decision  to  shift  runways  or  to  restrict  Orlando  departures  based  on 
TRACON/ARTCC  traffic  management  considerations. 
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IS  of  the  23  periods  involved  microburst  alerts.  Pilots  declined  to  land  or  to  take  off 
and  operations  ceased  on  the  impacted  runway  at  or  shortly  after  the  startup  of  the 
microburst  alerts.  In  some  cases,  operations  ceased  before  microburst  alert  startup 
due  to  the  factors  listed  above. 

An  Examination  of  the  Alerts  Issued  by  Local  Control  Identified  Two  Innovations  and  One 
Potential  Problem  Area 

An  examination  of  the  alerts  generated  by  the  system  and  the  alerts  verbally  issued  by  Lx)cal 
Control  to  landing/departing  pilots  indicates  that  157  air  crews  did  receive  or  should  have 
received  an  alert: 

25  of  the  157  air  crews  were  involved  with  situations  where  Local  Control  did  not 
issue  the  alert  in  effect  for  the  air  crew.  Almost  all  of  these  situations  involved  the 
rirst  one  or  more  air  crews  not  being  issued  an  alert  after  alert  startup.  This  number  of 
“non-issued”  alerts  may  warrant  a  re-examination  of  the  means  us»d  to  get  controller 
attention  at  alert  startup,  which  was  also  identified  as  a  problem  area  in  earlier 
demonstrations. 

41  of  the  157  air  crews  were  involved  with  situations  where  Local  Control  advised 
the  air  crew  of  an  alert  in  effect  for  some  adjacent  airspace  and  primarily  involved 
landing  air  crews  being  advised  of  an  alert  in  effect  for  Ae  airspace  off  the  departure 
end  of  the  arrival  runway.  The  innovation  of  “adjacent-airspace-alert”  advisories  as 
heads-up  information  to  pilots  in  the  event  of  a  missed  approach  appears  to  have  been 
well  received  by  the  involved  pilots  and  should  be  examined  for  possible  inclusion 
into  the  formal  guidelines  to  controllers  on  the  issuance  of  alerts  that  will  accompany 
the  introduction  of  the  operational  system. 

91  of  the  157  air  crews  were  involved  with  simations  where  Local  Control  issued  the 
appropriate  alert  that  was  directly  applicable  to  the  airspace  that  was  to  be  traversed 
during  landing  or  takeoff. 

The  second  innovation  involved  the  132  air  crews  that  received  either  a  “directly-applicable” 
alert  or  an  “adjacent-airspace-alert”  advisory  and,  specifically,  the  61  air  crews  that  received  an 
alen/advisory  that  located  the  wind  shear  on  the  runway.  In  an  apparent  attempt  to  provide  more 
detailed  location  information,  controllers  substituted  in  the  phrase  “approach  end  of  the  runway” 
or  “departure  end  of  the  runway”  in  issuing  47  [or  77  percent]  of  the  61  alerts/advisories  instead 
of  using  the  standard  phraseology  “over  the  runway.” 

Pilot  Utilization  of  Issued  Alerts 

Prior  to  the  demonstration,  some  of  the  airlines  operating  out  of  Orlando  were  known  to  have 
instructed  their  pilots  not  to  continue  their  landing  or  takeoff  on  receiving  a  microbiust  alert,  if  at 
all  possible. 

Table  A-1  characterizes  the  “directly-applicable”  alerts  verbally  issued  to  pilots  during  the  39 
alert  periods  in  terms  of  the  intensity  of  the  alert  issued  and  the  action  taken  by  the  pilot  as  to 
whether  the  landing/takeoff  was  completed  or  delayed.  The  table  shows  that:  (1)  few  “gain” 
wind  shear  alerts  were  issued,  as  one  would  expect  given  the  Orlando  wind  shear  environment, 
and  (2)  few  microburst  alerts  were  issued  directly  to  pilots,  as  one  would  expect  given  that 
runway  operations  ceased  at  microburst  alert  startup  or  shortly  thereafter.  Alert  updates 
broadcast  for  status  information  during  these  operational  pauses  and  not  directed  to  specific 
aircraft  actively  in  the  process  of  landing  or  taking  off  were  not  included  in  the  table.  In  terms  of 
wind  shear  avoidance,  1  (or  12  percent)  of  the  8  air  crews  issued  a  “gain”  wind  shear  alert 
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declined  to  land,  10  (or  14  percent)  of  the  70  air  crews  issued  a  “loss”  wind  shear  alert  declined 
to  land  or  take  off,  and  7  (or  54  percent)  of  the  13  air  crews  explicitly  issued  a  microburst  alen 
declined  to  land  or  take  off  with  the  alert  in  effect. 


Table  A>1 

Pilot  Utilization  of  “Directiy-Applicabie'*  Alerts 
Issued  to  Pilots  During  the  39  Examined  Alert  Periods 


Alert  Actually 
Issued  to  Pilot 

Number  of 
Air  Crews 
Issued  Such 
an  Alert 

+15kts 

3 

+20kts 

3 

+25kts 

2 

Operational  Outcome 

Pilot  Landed 

Pilot  Did  Not 
Complete 
Landing 

Pilot  Took 
Off 

Pilot  Delayed 
Takeoff 

Subtotal 


-ISkts 

12 

-20kts 

31 

-25kts 

2Z 

Subtotal 

70 

-30kts 


-35kts 


-40kts 


-45kts 


-SOkts 


-55kts 


Higher 


Subtotal 


TOTAL 


2 

5 

25 

10 
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Table  A-2  shows  the  corresponding  situation  for  the  issued  “adjacent-airspace-alert” 
advisories.  It  is  seen  that  6  of  the  10  air  crews  issued  a  microburst  alert  advisory  for  a  microburst 
in  some  adjacent  airspace  declined  to  land  or  take  off. 

Table  A-2 

Pilot  Utilization  of  “Adjacent-Airspace-Alert”  Advisories 
Issued  to  Pilots  During  the  39  Examined  Aiert  Periods 
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Table  A- 3  presents  the  operational  details  of  the  six  air  crews  that  proceeded  to  land  or  take 
off  after  receiving  a  “directly-applicable”  microburst  alert.  It  is  seen  from  the  table  that  four  of 
air  crews  proceeded  to  land  or  t^e  off  into  rapidly  improving  wind  shear  conditions  that  dropped 
to  wind  shear  status  within  a  minute  of  the  microburst  alert  being  issued  to  the  air  crew  and  that 
in  one  other  case  the  microburst  alert  was  a  false  alarm  due  to  a  faulty  LLWAS  sensor.  In  only 
one  of  these  five  cases  was  the  air  crew  possibly  notified  of  the  improving/correct  wind  shear 
situation.  This  apparent  ability  to  coirecdy  evaluate  the  wind  shear  situation  out  the  cockpit 
window  is  probably  due  to  the  Orlando  wind  shear  environment  which  permits  pilots  to  “see” 
microbursts  in  terms  of  blowing  precipitation  in  close  association  with  rain  shafts. 

However,  one  case  makes  the  point  that  this  out-the-window  capability  to  correctly  evaluate 
microburst  situations  is  less  than  perfect  among  Orlando  pilots.  After  landing,  one  pilot,  who 
had  been  issued  a  30-knot  microburst  alert,  advised  Local  Control  that  nobody  else  should  land 

to  which  the  local  controller  observed;  “You  sure  beat  that  one  by  a  whisker . just  as  you  got 

to  the  other  side  of  it,  we  had  a  microburst  alert  with  a  60-knot  loss  on  3-Mile  Final.” 


Table  A-3 

Details  of  the  Six  Air  Crews  that  Proceeded  to  Land  or  Take  Off 
After  Being  Issued  a  Microburst  Alert 


Runway, 
Greenwich 
Mean  Time, 
and  Date 
of  the 
Operation 

issued  Alert 

PIREP 

Content 

Details  of  the  Landing  or  Takeoff 

1  Runway 

36R  Arrival 
at  1809 

GMT  on 
6-15-92 

40-knot  loss 
1-Mile  Final 

Smooth,  ±10 
knots  at  200 
feet 

This  was  the  first  approach  to  Runway  36R  after  a 
mnwav  shift  durino  the  alert  oeriod.  Wind  shear 
conditions  were  imoroving  rapidly  on  Runway  36R  at 

the  time  in  that  alerts  dropped  to  wind  shear  status 
(-25KTS/RWY]  within  a  minute  of  the  pilot  receiving  the 
microburst  alert.  The  pilot  was  not  advised  of  the 
status  change  directly  but  may  have  overheard  the 
next  pilot  to  contact  Local  Control  for  landing  clearance 
receive  a  wind  shear  alert. 

2  Runway  17 
Arrival  at 
1646  GMT 
on  6-20-92 

30-knot  loss 
2-Mile  Final 

No  PIREP 

This  was  the  first  alert  issued  to  an  air  crew 
approaching  Runway  17  during  the  alert  period,  yvjnd 
shear  conditions  were  irrxiroving  in  that  alerts  dropped 

to  wind  shear  status  [-25KTS/2MF]  one  minute  after 
the  pilot  received  the  microburst  alert.  Local  Control 
did  not  advise  the  pilot  of  the  chanoe  in  status. 
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Table  A-3 
(Continued) 


Runway, 
Greenwich 
Mean  Time, 
and  Date 
of  the 
Ooeratlon 

iasued  Alert 

PIREP 

Content 

Details  of  the  Landing  or  Takeoff 

3  Runway 

36R 

Departure 
at  1656 

GMT  on 
6-20-92 

35-knot  loss 
Departure 

End 

No  PIREP 

This  was  the  first  departure  from  Runway  36R  after  a 
runwav  shift  durino  the  alert  oeriod.  Wind  shear 
conditions  were  imoroving  rapidly  on  Runway  36R  in 

that  alerts  dropped  to  wind  shear  status  [-20kTS/RWY] 
one  minute  after  the  pilot  received  the  microburst  alert. 

In  addition,  the  pilot  turned  sharply  after  takeoff  in 
order  to  avoid  the  rain  shaft  at  the  departure  end  of  the 
runway.  Controller  arxJ  pilots  commonly  use  rain 
shafts  as  markers  to  locate  microbursts  at  Orlando 
International  Airport. 

4  Runway 

18R  Arrival 
at  1837 

GMT  on 
6-23-92 

35-knot  loss 
over  runway 

No  PIREP 

This  was  a  false  alert  caused  bv  a  bad  LLWAS  sensor 
as  noted  in  the  daily  log  maintained  by  the  on-site 
demonstration  team,  the  LLWAS  channel  was 
temporarily  taken  out  of  senrice  shortly  after  this 
aircraft  landed. 

5  Runway  17 
Departure 
at  21 19 

GMT  on 
7-06-92 

35-knot  loss 
Departure 

End 

No  PIREP 

This  was  the  first  departure  from  Runway  17  after 
departure  operations  had  ceased  for  8  minutes  due  to 
microburst  alerts  being  in  effect  and  other 
thunderstorm  activity  in  departure  airspace.  Wind 
shear  conditions  were  imorovinq  rapidiv  at  the  time  on 

Runway  17  in  that  alerts  dropped  to  wind  shear  status 
[-25KTS/RWY]  within  a  minute  of  the  pilot  receiving  the 
microburst  alert.  In  addition,  the  pilot  turrved  sharply 
after  takeoff  in  order  to  avoid  the  rain  shaft  and 
associated  wind  shear  at  the  end  of  the  departure  end 
of  the  runway. 

6  Runway  17 
Arrivai  at 
2044  GMT 
on  7-07-92 

25-knot  loss 
3-Mile  Final 

Updated  to 
30-knot  loss 
3-Mile  Final 

85  sec.  later 

Advise 
nobody  land 
on  17 

The  alert  period  started  seconds  before  this  pilot 
reauested  landina  clearance.  Wind  shear  conditions 
were  worsening  rapidly  during  the  approach  as 

evidenced  by  the  two  alerts  issued  to  the  pilot  and  the 
comment  made  by  Local  Control  to  the  pilot  shortly 
after  landing  ‘You  sure  beat  that  one  by  a 
whisker... just  as  you  got  to  the  other  side  of  it,  we  had 
a  microburst  alert  with  a  60-knot  loss  at  3-Mile  Final.” 
Arrival  operations  ceased  after  this  landing. 
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In  contrast  to  earlier  demonstrations,  this  demonstration  suggests  that  more  pilots  are  seeking 
to  avoid  microburst  encounters.  For  example,  during  the  199H0  Orlando  TDWR  Demonstration 
pilots  took  off  or  landed  after  being  issued  a  microburst  alert  once  every  one  to  two  days,  on 
average  [i.e.,  29  air  crews  did  so  over  the  37-day  demonstration].  However,  during  the  1992 
Demonstration,  this  average  was  reduced  to  roughly  once  every  ten  days  [i.e.,  six  air  crews 
landed  or  took  off  after  being  issued  a  microburst  dert  over  the  65-day  demonstration]. 

Pilot  use  of  issued  alerts  for  microburst  avoidance  is  key  if  the  envisioned  safety  benefit  is  to 
be  realized  by  the  deployed  system.  The  1992  Demonstration  suggests  a  trend  that  favors  the 
future  realization  of  the  system’s  envisioned  safety  benefit 

System  Performed  Well  Overall  Based  on  Pilot  Reports 

PIREPs  of  the  wind  and  weather  conditions  observed/experienced  by  pilots  were  received 
from  31  of  the  91  air  crews  that  received  a  “diiectly-applicable”  alert  and  from  an  additional  74 
landing/departing  air  crews  during  the  39  examined  alert  periods.  Based  on  these  105  PIREPs, 
the  system,  over^U,  provided  landing/departing  pilots  with  timely  alert  coverage  of  all  significant 
wind  shear  conditions. 

TDWR  and  the  Integrated  TDWR/LLWAS  alert  systems  were  not  designed  to  provide  alert 
coverage  for  all  wind-related  encounters  experienced  by  landing/departing  pilots.  Table  A-4 
characterizes  the  four  encounters  not  provided  alert  coverage  during  the  39  examined  alert 
periods.  A  review  of  the  Doppler  and  mesonet  data  indicated  that  the  encounters  involved  two 
wind  conditions  not  provided  alert  coverage:  (1)  thermals,  and  (2)  periods  of  strong,  gusty 
surface  winds  that  are  not  in  close  association  with  an  area  of  organized  with  shear  (e.g.,  a  gust 
front  or  microburst). 
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Table  A<4 

Four  Encounters  Reported  by  Pilots  that  Were  Not  Provided 
Alert  Coverage  by  the  Tested  System 
During  the  39  Examined  Alert  Periods 


Runway,  Greenwich 
Mean  Time  and  Date 
of  the  Operation 
(1)(2) 

PIREP  Content 

“Best  Guess”  Explanation  as  to  Why  Alert 
Coverage  Was  Not  Provided  In  This  Case 
(3) 

1  Pilot  landed  on 
Runway  17  at  1551 
GMT  on  6-05-92 

Winds  are  real  strong... 
the  following  pilot  may  want  to 
go  around 

A  review  of  the  Doppler  data  suggests  that  the 
aircraft  encountered  strong,  turbulent,  surface 
aoss  winds  associated  with  a  macroburst's 
outflow.  The  airport’s  centerfield  wind  at  the 
time  of  the  encounter  was  240  degrees  at 

15  knots,  gusting  to  40  knots.  The  system  did 
not  generate  alerts  in  this  case  because  a 
microburst  was  not  involved  and  the  outer 
edge  of  the  outflow  did  not  trigger  gust-front- 
based  alerts  for  it  was  delineated  by  a  broad 
area  of  slowly  increasing  wind  speeds  rather 
than  by  the  narrow  band  of  sharply  increasing 
wind  speeds  [i.e.,  wind  shear]  that  typically 
defines  a  oust  front. 

2  Pilot  landed  on 
Runway  l8Rat 

2220  GMT  on 

Pretty  good  sinker  at  800  feet 

A  review  of  the  Doppler  data  suggests  that  this 
and  the  next  aircraft  encountered  a  weak  gust 
front  outflow  from  a  distant  microburst,  not  an 

6-20-92 

ITl  (Tpl 

3  Pilot  landed  on 
Runway  18Rat 

2222  GMT  on 
6-20-92 

Gained  1 5  to  20  knots  at 

800  feet 

See  the  above  entry. 

4  Pilot  took  off  from 
Runway  18Lat 

2244  GMT  on 
6-30-92 

Lost  15  to  20  knots  at 

1000  feet 

Nothing  was  seen  in  the  Doppler  data 
suggesting  that  the  aircraft  encountered  some 
sort  of  small-scale,  wind  shear  feature  [e.g.,  a 
thermall.  not  an  alert  situation. 

Note:  (1 )  Results  are  based  on  a  review  of  the  39  examined  alert  periods  for  those  pilots  for  which  an 
alert  was  not  generated  yet  reported  a  relatively  intense  encounter  (i.e.,  an  airspeed 
variation  in  excess  of  15  knots,  moderate  or  greater  turbulence,  any  indication  of  a 
downflow,  or  any  suggestion  that  extra  caution  was  advisable). 

(2)  The  Greenwich  Mean  Time  (GMT)  refers  to  the  time  that  Local  Control  issued 
landing/takeoff  clearance  to  the  pilot. 

(3)  These  explanations  are  based  on  discussions  with  Lincoln  Laboratory  personnel  who  had 
access  to  the  stored  Doppler  weather  radar  and  Mesonet  data  for  the  pertinent  periods. 
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A  “WHAT  IF’  COMPARISON  OF  FOUR  ALERT  STRATEGIES  RELATIVE  TO 
THEIR  IMPACT  OVER  THE  39  EXAMINED  ALERT  PERIODS 


Four  alert  systems/softwares  were  examined  relative  to  what  their  operational  impact  would 
have  been  over  the  39  examined  alert  periods:  (1)  the  Integrated  TDWR/LLWAS  System,  (2) 
the  TDWR  channel  of  the  Integrated  System  as  a  stand-alone  alert  system,  (3)  the  LLWAS 
channel  as  a  stand-alone  alert  system,  and  (4)  TDWR  as  a  stand-alone  alert  system  but  using  the 
alert  software  as  it  existed  in  1990.  Table  A-5  compares  the  four  alternatives  in  terms  of  the 
overall  system  alert  activity  that  would  have  occurred.  The  left  side  of  the  table  compares  the 
two  versions  of  TDWR  as  stand-alone  alert  systems  to  the  Integrated  TDWR/LLWAS  system 
and  shows  an  overall  decrease  in  alert  activity.  The  right  side  of  Ae  table  compares  LLWAS  as  a 
stand-alone  alert  system  to  the  Integrated  TDWR/LLWAS  system  and  shows  an  overall  increase 
in  alert  activity. 

The  earliest  form  of  the  alert  systems/softwares  examined  was  TDWR  as  a  stand-alone 
system  using  1990  alert  software.  Major  modifications  were  made  to  the  TDWR  software  after 
1990  to  reduce  the  extent  that  “nuisance”  alerts  occurred  and  to  improve  the  overall  accuracy  of 
the  alerts  issued  to  pilots.  It  is  seen  from  Table  A-5  that  this  alternative  would  have  maximized 
alert  activity  during  the  1992  IDemonstration  in  terms  of  the  amount  of  time  that  the  runways 
were  under  alert  status  at  808  minutes  or  13.5  hours  and  the  number  of  air  crews  issued  an  alert 
at  233  air  crews. 

The  second  alternative  was  TDWR  as  a  stand-alone  alert  system  using  1992  software.  In  the 
1992  Demonstration,  use  of  TDWR  as  a  stand-alone  alert  system  with  1992  versus  1990  software 
would  have  reduced  alert  activity  by:  (1)  21  percent  in  terms  of  overall  alert  duration  [10.7 
versus  13.5  hours],  (2)  36  percent  in  terms  of  the  number  of  air  crews  for  which  an  alert  was 
generated  [148  versus  233  air  crews],  and  (3)  68  percent  in  terms  of  the  number  of  air  crews  for 
which  a  microburst  alert  was  generated  [21  versus  66  air  crews]. 

PIREPs  support  the  reduction  in  alert  activity  in  that  the  worst-case  encounter  reported  by 
those  pilots  that  would  have  received:  (1)  an  alert  with  the  1990  software  but  not  with  the  1992 
software  was  for  a  “15-knot  gain,”  a  threshold-level  encounter,  or  (2)  a  microburst  alert  with  the 
1990  software  but  a  “loss”  wind  shear  alert  with  the  1992  software  was  for  a  “20-knot  gain,”  a 
less-than-microburst-intensity  encounter. 

The  third  alternative  was  the  LLWAS  channel  of  the  integrated  TDWR/LLWAS  alert  system 
as  a  stand-alone  alert  system  using  1992  software.  It  is  seen  from  Table  A-5  that  this  alternative 
would  have  minimized  alert  activity  during  the  1992  Demonstration  in  terms  of  the  overall 
amount  of  time  that  the  runways  were  under  alert  status  at  5.7  hours  and  the  number  of 
operations  issued  an  alert  at  87  air  crews.  This  “minimum”  finding  may  not  be  representative  of 
airports  with  a  high  ratio  of  gust  fronts  to  microbursts  relative  to  Orlando’s  wind  shear 
environment,  such  as  at  Denver’s  Stapleton  International  Airport. 
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Table  A-5 

Alert  System/Software  Comparison  Relative  to  Alert  Activity 


Alert  Activity 

1990  TDWR 
Alert  Software 

1992  TDWR 
Alert  Software 

1992  Integrated 
TDWRALWAS 
Alert  Software 

1992  LLWAS 
Alert  Software 

1992  Integrated 
TDWR/LLWAS 
Alert  Software 

Alert  Periods 

Number 

39  alert  periods 

37 

40 

33 

40 

Duration 

Total 

808  minutes 

640 

628 

342 

628 

2  to  68  mins. 

1  to  42 

1  to  37 

1  to  37 

Average/ 

period 

20  mins7period 

17 

15 

10 

15 

Average/ 

week 

90  minsAweek 

71 

70 

38 

70 

Number  of  Air 
Crews  That 
Would  Have 
Rsce'ived  an 

Alert 

Total  Number 

233  air  crews 

148 

141 

87 

141 

Microburst  Alert 

66  air  crews 

21 

21 

14 

21 

“Loss"  Wind 
Shear  Alert 

141  aircrews 

101 

92 

40 

92 

“Gain”  Wind 
Shear  Alert 

26  air  crews 

26 

28 

33 

28 

The  fourth  and  final  alternative  was  the  integrated  TDWR/LLWAS  alert  system  using  the 
1992  software.  Tables  A-6  and  A-7  present  the  impact  of  integration  over  what  the  individual 
TDWR  and  LLWAS  channels  would  have  done  as  stand-alone  alert  systems  over  the  39 
examined  alert  periods.  From  the  tables,  one  sees  the  extent  that  integration  increased  alert 
coverage,  improved  alert  timeliness,  decreased  extraneous  alerts,  and  improved  alert  accuracy 
relative  to  what  either  TDWR  or  LLWAS  would  have  done  alone. 
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Table  A-6 

Impact  of  TDWR/LLWAS  Integration 
on  Alerts  Generated  by  the  TDWR  Channel 


Envisioned  impact  of  integration 
on  TDWR-based  Aierts 

The  “What  ir’  Results  Relative  to  the 

39  Examined  Alert  Periods 

1 .  Reduction  in  potential  nuisance 
alerts  p.e.,  elimination  of  all  “gain” 
wind  shear  aierts  and  15-knot 
“loss”  wind  shear  alerts  not 
confirmed  by  the  LLWAS 
channel] 

34  air  crews  that  would  have  received  a  TDWR-based 
alert  would  not  have  received  an  integrated  alert... the 
PIREPs  support  the  elimination  of  these  alerts  in  that  the 
most  intense  encounter  reported  by  this  group  was  for  a 
“10-knot  gain  last  400  feet,”  a  non-alert  situation. 

2.  More  accurate  alerts  involving 
gust '  nts,  microbursts  and 
outf’  s  that  could  intensify  to 
mic.'Oburst  alert  status 

23  air  crews  that  would  have  received  a  TDWR-based 
alert  would  also  have  received  an  integrated  alert  but 
with  some  difference  in  terms  of  the  wind  shear  intensity 
and/or  location  estimates... the  PIREPs  only  provided 
sufficient  detail  in  one  case  for  a  comparison  and  that 
case  supports  the  downgrading  of  a  TDWR-based, 
microburst  alert  to  an  integrated,  “loss”  wind  shear  alert. 
The  PIREP  was  for  “10-knot  gain,  then  a  12-knot  loss 
and  strong  lateral  winds.” 

3.  Expanded  alert  coverage  of  gust 
fronts 

20  air  crews  that  would  not  have  received  a  TDWR- 
based  alert  would  have  received  an  integrated,  “gain” 
wind  shear  alert.. .the  PIREPs  tend  to  support  the 
integrated  alerts  [i.e.,  "gained  20  knots  on  Short  Final” 
and  “30-knot  tail  wind,  going  around”]  but  there  were 
also  four  PIREPs  indicating  something  to  the  effect  that 
“little  or  nothing  was  encountered.” 

4.  Expanded  alert  coverage  of 
outflows  that  could  intensify  to 
microburst  alert  status 

7  air  aews  that  would  not  have  received  a  TDWR-based 
alert  would  have  received  an  integrated,  “loss”  wind 
shear  aiert...the  sole  PIREP  received  from  this  group 
does  not  support  the  integrated  alerts  in  this  case  in  that 
the  pilot  reported  “no  change  in  airspeed.” 

5.  More  timely  alerts 

1  air  crew  would  have  received  a  more  timely  integrated 
alert  in  the  form  of  a  “gain”  wind  shear  alert  versus  a  later 
TDWR-based  alert... .the  PIREP  received  from  this  pilot 
supports  the  integrated  alert  "gained  12  to  15  knots  at 

450  feet.” 

6.  A  resulting  change  in  overall  alert 
activity 

2%  decrease  in  overall  alert  duration  [10.5  versus  10.7 
hours]  and  a  5%  decrease  in  terms  of  the  number  of  air 
crews  issued  an  alert  [141  versus  148  air  crews].  These 
modest  decreases  reflect  the  low  ratio  of  gust  fronts  to 
microbursts  impacting  Orlando  International  Airport. 
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Table  A-7 

Impact  of  TDWR/LLWAS  Integration 
on  Alerts  Generated  by  the  LLWAS  Channel 


Envisioned  Impact  of  imegration 
on  LLWAS*based  Alerts 

The  “What  ir  Results  Relative  to  the 

39  Examined  Alert  Periods 

1 .  Expanded  alert  coverage  of 
microbursts  and  outflows  that 
could  intensify  to  microburst  alert 
status. 

63  air  crews  that  would  not  have  received  an  LLWAS- 
based  aiert  would  have  received  an  integrated  "loss" 
wind  shear  aiert  [59  air  crews]  or  a  microburst  alert  [4  air 
crews]. ..PIREP  support  is  mixed  for  the  integrated  alerts 
from  these  air  crews  [i.e.,  3  air  crews  reported  significant 
encounters,  such  as  "gained  20  knots,”  versus  12  air 
crews  that  reported  somethino  to  the  effect  that  "iittie  or 
nothing  was  encountered"].  The  mixed  PIREP  support 
might  reflect  the  wet  atmospheric  conditions  at  Orlando 
that  permit  pilots  to  frequently  use  rain  shafts  as  markers 
to  locate  microbursts  in  making  their  go/no-go  landing 
and  takeoff  decisions.  Note  that  1 2  of  the  63  air  crews 
chose  either  to  delay  their  takeoff  or  to  go  around. 

2.  More  accurate  alerts  involving 
microbursts  and  outflows  that 
could  intensify  to  microburst  alert 
status. 

33  air  crews  that  wouid  have  received  an  LLWAS-based 
alert  would  ^so  have  received  an  integrated  alert  but 
with  some  difference  in  terms  of  the:  (a)  wind  shear 
intensity  estimate,  (b)  wind  shear  location  estimate, 
and/or  (c)  the  type  of  wind  shear  involved... the  PIREPs 
do  not  provide  sufficient  detail  for  an  evaluation  of  the 
differences  between  the  LLWAS-based  and  integrated 
alerts. 

3.  More  timely  alerts  involving 
microbursts  and  outflows  that 
could  intensify  to  microburst  alert 
status. 

1 1  air  crews  would  have  received  a  more  timely 
ntegrated  alert  in  the  form  of  a  "loss"  wind  shear  aiert 
[6  air  crews]  or  a  microburst  alert  [5  air  crews]  versus  a 
ater  LLWAS-based  alert.. .pilot  actions  and  PIREPs 
support  the  integrated  alerts  in  that  5  air  crews  [3  issued 
a  microburst  aiert  and  2  issued  a  "loss"  wind  shear  alert] 
chose  to  either  delay  takeoff  or  to  go  around  and  two 
pilots  landed  and  reported  "advise  that  nobody  land"  and 
"shear  at  300  feet." 

4.  Reduction  in  potential  nuisance 
alerts  [i.e.,  elimination  of  15-knot 
"loss”  wind  shear  alerts  not 
confirmed  by  the  TDWR 
channel]. 

9  air  crews  that  would  have  received  a  15-knot,  LLWAS- 
based  alert  would  not  have  received  an  integrated 
alert... ttie  sole  PIREP  from  this  group  supports  the 
elimination  of  these  alerts,  "no  airspeed  variation,  smooth 
ride." 

5.  A  resulting  change  in  overall  alert 
activity. 

84%  increase  observed  in  terms  of  overall  alert  duration 
[10.5  versus  5.7  hours]  and  a  62%  increase  observed  in 
terms  of  the  number  of  air  crews  that  wouid  have  been 
issued  an  aiert  [141  versus  87  air  crews].  These 
increases  reflect  the  high  ratio  of  microbursts  to  gust 
fronts  impacting  Orlando  International  Airport. 
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One  also  finds  in  the  tables  that  the  PIREPs  tend  to  support  the  extensive  changes  introduced 
by  integration  overall  but  that  the  support  is  modest  to  nonexistent  in  certain  instances.  It  is 
suggested  that  this  is  due  to  two  factors  related  specifically  to  Orlando  and  the  1992 
Demonstration:  (1)  Orlando  pilots  experience  and  report  few  significant  wind  shear  encounters 
relative  to  other  airports,  such  as  Stapleton  International  Airport,  because  the  pilots  tend  to  be 
skilled  at  using  the  visual  cues  available  in  the  wet,  Orlando  environment  to  “see”  and  avoid 
microbursts  and  outflows  that  could  increase  to  nucroburst  alert  status,  and  (2)  the  1992 
Demonstration  provided  surprisingly  few  PIREPs  overall  with  which  to  evaluate  the  alternative 
alert  systems/softwares. 

LOST  PIREPs 

It  is  clear  from  the  communication  tape  analysis  done  for  this  smdy  that  the  taping  method 
used  during  the  demonstration  did  not  record  all  the  PIREPs  provided  by  landing  pilots.  The 
taping  was  done  at  the  TDWR  site,  which  was  located  some  miles  from  the  airport.  Apparently, 
there  were  line-of- sight  problems  between  the  radio  antenna  at  the  radar  site  and  portions  of  the 
airport’s  runways  and  taxiways.  At  times  throughout  the  demonstration.  Local  Control  could  be 
heard  on  the  communication  tapes  requesting  PIREPs  from  pilots  that  had  just  landed,  but  no 
replies  could  be  heard  except  for  the  pilot  keying  the  microphone  in  making  a  response.  Based  on 
listening  to  the  tapes,  it  is  believed  that  a  substantial  number  of  PIREPs  were  lost  during  the 
1992  Demonstration. 

If  communication  tapes  are  to  be  saved  for  possible  analysis  during  future  demonstrations, 
the  issue  of  “lost  PIREPs”  due  to  off-airport  taping  of  Local  Control-pilot  radio  communications 
should  be  addressed. 


